search

zyainfal / One-Shot-Face-Swapping-on-Megapixels

One Shot Face Swapping on Megapixels.
Other
314 stars 40 forks source link

About the training result #23

Closed jo-dean closed 2 years ago

jo-dean commented 2 years ago

image As shown in above img,the swaped face in the top looks bad,and the bottom is good. Did you know why this happens? Thank you!

zyainfal commented 2 years ago

The reason is rooted in HieRFE. As we stated in ablation study, different design of latent space gives different generalization ability (shown by failure rate), which means if HieRFE fails to generate a good latent code for reconstruction, it is impossible for FTM to swap the faces.

Usually, more real images can improve the generalization ability of the network.

zyainfal commented 2 years ago

Besides, for GAN inversion part, i.e. what HieRFE does, you can use optimization algorithms (rather than a neural network), which are bound to give at least a decent latent code.

However, I guess it could be difficult for optimization algorithms to generate a good constant input of 4x4x512.

jo-dean commented 2 years ago

thank you for your reply!