search

felixrosberg / FaceDancer

Other
465 stars 78 forks source link

Is there a way to retain the facial marks of the source image. #31

Closed Suvi-dha closed 1 year ago

felixrosberg commented 1 year ago

Do you mean saving the landmarks? Or visualizing? You can see in the swap code the landmarks being prepared. You should be able to add what you need to that.

Suvi-dha commented 1 year ago

Sorry I did not describe the problem. I mean the facial marks like spots or freckles on the skin of the source image.

felixrosberg commented 1 year ago

Ah okey. Well freckles and spots are quite small, and these kind of models struggle with small details like that. It is also regularized towards keeping semantics in the target face such as make up. I wouldn't be surprised if it prioritizes freckles on the target sometime because of this. We also rely on ArcFace to extract the necessary identity information, which demands a 112x112 image, which is rather small. I would assume ArcFace does not really care about freckles by that point.

Perhaps FaceDancer can be improved by using an ArcFace trained on higher resolution images and utilizing low-level features in some way. How and if this works I have no idea. And if it did work it is no guarantee for freckles.

Suvi-dha commented 1 year ago

Ok got it. That's another problem statement! I'll see if possible, whether during training I can finetune the regularization parameter to dull the semantics of the target face and focus on source face. Thanks a lot for your time.

arnold408 commented 1 year ago

112x112 image

Does ArcFace provide a bigger model?