search

eladrich / pixel2style2pixel

Official Implementation for "Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation" (CVPR 2021) presenting the pixel2style2pixel (pSp) framework
https://eladrich.github.io/pixel2style2pixel/
MIT License
3.2k stars 568 forks source link

Toonify without a smile #248

Closed mohammad-rj closed 2 years ago

mohammad-rj commented 2 years ago

Thank you for the interesting work.

image

is it possible use toonify photo without smileing ?

image I need the output like this result

I'm not very familiar with these issues (StyleGAN - stylegan_weights - ffhq_cartoon_blended.pt - psp_ffhq_encode.pt - psp_ffhq_toonify.pt - stylegan2-ffhq-config-f.pkl - ffhq - training - models - dataset - res - targets - test and ... ) Is it possible for you guys make a colab for this particular purpose plz ? or guide me how I can edit this colab to do this ?

yuval-alaluf commented 2 years ago

The toonify StyleGAN generator has a bias for outputting images with a smile since that is the data it was trained on. What you can do is encode the image and then edit it to remove the smile. Although I don't have the time to make a notebook to demonstrate editing, there are a few scripts for a couple similar repos that may be able to help you out:

  1. https://github.com/omertov/encoder4editing/blob/main/editings/latent_editor.py Here the latent editor can get a latent representing the image encoding and a direction smile and you can edit the latent representation according to that direction.
  2. https://github.com/yuval-alaluf/restyle-encoder#editing In another one of my repos, I provide more details and a script for performing editing on real images. Some changes are required, but this has 80% of what you need.