ISSUE: Latest version Face Mask show BOX EDGES (no blur)

[AlonDan]

I've downloaded the latest version and when it's finished rendering via the "Animate" button Sometimes everything looks fine, but sometimes there are very obvious EDGES around the face that looks like the 1:1 ratio BOX, I can't say when it happens but I sure noticed it in some cases.

In my tests I used the default settings and just pressed "Animate" I didn't touch any other slider.

My guess is that it's doing something with the resize of the stitching part and maybe the mask process BEFORE instead of after, so it's not blending well / smooth enough for the FINAL video result.

It will be nice to give the user the control (via SLIDER) for the amount BLUR on the edges of the stitched face on the image.

---

Couple extra questions:

1 - Will it be possible to select SPECIFC faces, or MULTIPLE faces like in your paper examples?

2 - Same for video, will it be possible to Drag & Drop VIDEO on the Gradio App anytime soon? That will be nice to test and give you feedback of course 👍

Thanks ahead for your wonderful work on this project 🙏

[zzzweakman]

I've downloaded the latest version and when it's finished rendering via the "Animate" button Sometimes everything looks fine, but sometimes there are very obvious EDGES around the face that looks like the 1:1 ratio BOX, I can't say when it happens but I sure noticed it in some cases.

In my tests I used the default settings and just pressed "Animate" I didn't touch any other slider.

My guess is that it's doing something with the resize of the stitching part and maybe the mask process BEFORE instead of after, so it's not blending well / smooth enough for the FINAL video result.

It will be nice to give the user the control (via SLIDER) for the amount BLUR on the edges of the stitched face on the image.

Couple extra questions:

1 - Will it be possible to select SPECIFC faces, or MULTIPLE faces like in your paper examples?

2 - Same for video, will it be possible to Drag & Drop VIDEO on the Gradio App anytime soon? That will be nice to test and give you feedback of course 👍

Thanks ahead for your wonderful work on this project 🙏

Thank you for your feedback. This is a known issue, and we are actively working on a solution. If you have any suggestions, we would love to discuss them and welcome any PR : )

Regarding your observations, could you please share the images and videos where you noticed the obvious edges?

To answer your questions:

1. Currently, we automatically select the first face returned by the cropper. We are considering adding a parameter to allow users to select specific or multiple faces in a future version.
2. Are you referring to uploading the driving video? The Gradio App now supports Drag & Drop for videos.

[AlonDan]

Thank you for the quick reply, Since I couldn't reproduce while testing different random videos I couldn't reproduce where exactly the edges are more obvious so I will keep try until I'll find a specific example so I can share with you.

1. Selecting Multiple / Specific faces from the source will be amazing! I hope you will add this.

2. I was talking about that the Gradio only accept IMAGES on source, while I was trying to test video like in your paper examples, it looks amazing and I'm curious how it will look in some different situations / angles / starting face / camera movement etc.. via video is probably more challenging than an image but the VIDEO as SOURCE in general is sure more interesting to tests.

I hope that you will consider to add the VIDEO feature, what I can recommend is if and when you will add this option an AUTOMATIC FPS recognize once the SOURCE VIDEO will be dragged to it's container in the Gradio App, it will be very nice and help to test and give more feedback on other potential issues.

Once again, thank you so much for your hard work I'm looking forward the VIDEO as source and hopefully the Multiple / Specific faces from source as well, I'll do my best to give more feedback in order to help improve upcoming versions 👍

[cleardusk]

Thanks for your feedback, @AlonDan . We will carefully consider these demands, but it may take a considerable amount of time to evaluate and implement them.