If you knock somebody to the ground, don't swoop in and try to melee them again as they are invunerable to melee during the period they have stopped moving and are recovering. In other words, you can't hurt them. Now, if they are still rolling around on the ground, it's fair game still
Also, if you are too close to your target (ie right next to them), the swoop will not activate. You have to put some distance between yourselfs. Anytime you see someone recover, don't try to melee them because you'll just screw yourselves over because they will be invunerable to melee. Hope this helps.