search

JohnnyRacer / subswap

Improved implementation of faceswap.
Apache License 2.0
3 stars 0 forks source link

Training question #1

Open antongonz opened 2 years ago

antongonz commented 2 years ago

Hello,

Awesome job of this! Very impressive!

In the tips you said "In case of finetuning model you can variate losses coefficients to make result look more like source identity, or vice versa, save features and attributes of target face."

How exactly do I do this?

Thanks!

JohnnyRacer commented 2 years ago

I've only been testing adaptive losses for a short while, and the code isn't really ready yet. But the implementation is basically driving the expected loss based on the similarity from arcface. As similarity is increased, the loss factors are decreased until clamped at a set value. This attempts to prevent the generator from overtly generating detail where it is not needed and leading to double eyebrows or irises. From the results I've got so far, this works well only when both the faces are of similar skin tone. If they are of significantly differing skintones, it tends to generate some strange and highly inconsistently results. By finetuning on a more varied dataset with differing ethnicities and lighting, the network should build a better understanding of how to transform adapted losses correctly. Hope this helps.