search

OpenTalker / StyleHEAT

[ECCV 2022] StyleHEAT: A framework for high-resolution editable talking face generation
MIT License
627 stars 77 forks source link

Cross-Identity Reenactment results not so good? #12

Closed c1a1o1 closed 1 year ago

c1a1o1 commented 2 years ago

Cross-Identity Reenactment results not so good? 40

https://user-images.githubusercontent.com/7991905/180114285-89b550da-150a-4a07-a9fd-29715211ba8e.mp4

FeiiYin commented 2 years ago

I cannot load the videos correctly to see the results. But indeed, the GAN inversion method is not perfect and the warping procedure will raise artifacts in the cross-id scenarios. The artifacts may be solved in the future.

Shotsordsoo commented 1 year ago

Can't load videos on post either, tried download image and run cross-id reenactment on my own system, worked nicely for me. https://user-images.githubusercontent.com/16696685/206864685-bebda9f8-53c4-49dd-997c-577467779ef9.mp4

FeiiYin commented 1 year ago

Ok, now I can watch the videos. I recommend using the optimization-based inversion methods, e.g. PTI or BaberShop to inverse the target identity. The identity will be strongly improved compared with the encoder-based inversion methods. The cross-id scenario will distort the original face, which is due to the incomplete decoupled identity information introduced by 3dmm. Incorporating information-bottleneck blocks to filter the information may help.