search

eladrich / pixel2style2pixel

Official Implementation for "Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation" (CVPR 2021) presenting the pixel2style2pixel (pSp) framework
https://eladrich.github.io/pixel2style2pixel/
MIT License
3.19k stars 570 forks source link

Is it possible to train pSp model on paired images? #58

Closed Nivratti closed 3 years ago

Nivratti commented 3 years ago

Thank you for your great work!!

I am training model to swap gender. It looks like your encoder is much faster. I have found this dataset synthesized using stylegan2.

https://www.dropbox.com/sh/pw6jn9sghwasylt/AADJjK2pvOAubW20hfAkMbqma/00571-gender-dataset-selected?dl=0&subfolder_nav_tracking=1

dataset contains paired images i.e. male -> female
and female-> male transformation

I have some few questions:

1) Is is possible to train pSp model on paired images? 2) What will be dataset structure? 3) Can I train pSp model on 512px resolution?

I know those questions may be trivial for you but it is vital for me to start the training.

yuval-alaluf commented 3 years ago

Hi @Nivratti ,

  1. pSp was designed to be trained on paired images. For example, look at our applications for sketch-to-image, segmentation-to-image, and super resolution to name a few applications using paired data.
  2. Our README explains how to set up your data: https://github.com/eladrich/pixel2style2pixel#preparing-your-dat You can also look at how we defined our experiments for the other paired applications.
  3. You can train pSp with 512x512 resolution with some small changes to the encoder code. However, you may find that training on 256x256 will give you just as good results with much faster training time and larger batch size. This is because the StyleGAN generator will still output 1024x1024 resolution images in either case.
Nivratti commented 3 years ago

Thanks Yuval for your quick reply.. I will follow your guidelines and I will start model training.