search

williamyang1991 / VToonify

[SIGGRAPH Asia 2022] VToonify: Controllable High-Resolution Portrait Video Style Transfer
Other
3.53k stars 446 forks source link

Edit face attributes [expressions: angry, sad, smiling] #33

Closed waleedrazakhan92 closed 1 year ago

waleedrazakhan92 commented 1 year ago

Hello, first of all thank you for such a wonderful work. I'd like to know if there's any way to manipulate face attributes like smiling, sad face, angry face, by using pretrained directions. As I've come to know that these boundaries are usually trained on PRE-PROCESSED images on either FFHQ dataset. But in your directory the faces are not cropped and aligned the same as FFHQ. So I'm wondering what do i need to do to be able to make cartoonization as well as manipulate further facial features. Your guidance is very highly appreciated

williamyang1991 commented 1 year ago

Our method removes the low-resolution layers of StyleGAN. So I'm afraid that our method cannot directly perform latent-editing with directions as the original StyleGAN. I think the best way is to perform video face attribute editing over the original face video using methods like https://yuval-alaluf.github.io/stylegan3-editing/ and https://github.com/IGLICT/DeepFaceVideoEditing Then, apply our method to the edited face for toonification.

waleedrazakhan92 commented 1 year ago

Thankyou so much for a quick reply. I'll embed the suggested changes in my pipeline.