Usage of reenact.py vs swap.py

[amil-rp-work]

Hey @YuvalNirkin Thanks for providing such a wonderful algorithm!

I tried to use swap.py and the face-swapping did happen but wasn't satisfactory. I believe the facial attributes like eyes, nose, mouth, etc are swapped but things like face shape, forehead size, and distance between the attributes are not fully replicated. Do you have any suggestions for me? (Have attached source, target and result as well) fsgan_outputs.zip The following command was used: python swap.py source.mp4 -t target.mp4 -o . -ec mp4v --finetune --finetune_save --seg_remove_mouth
I then move to face reenactment using reenact.py. However, I am unable to find a wiki to help me with the best possible output. Also, the output just has a cropped face. Any codebase you can point to that will help me combine the cropped face with the source video?

[YuvalNirkin]

Overall changing the shape of the target face is not very well defined. This face swapping methods follows the face landmarks of the target face and therefore does not preserve the facial shape of the source face. You can try manipulating the face landmarks or interpolating between them.

The same goes for pose and expression reenactment (such as the one implemented here), combining the reenactment with the full frame is not so simple.

[amil-rp-work]

Understood @YuvalNirkin, thanks for the comments. What are your suggestions for improving and making the landmark swap seem more natural?

• Like image/video quality
• Similar face characteristics etc?

For combining the reenactment with the full frame

• Can't we just identify the cropped part and place the final obtained video onto the original video at the same cropped region?

[shehrum]

Hi @amil-rp-work . Did you have any luck figuring out for the reenactment part i.e. Combining the reenactment result with the original source frame? I'm trying to figure this out. Any pointers/help would be appreciated @YuvalNirkin