search

eladrich / pixel2style2pixel

Official Implementation for "Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation" (CVPR 2021) presenting the pixel2style2pixel (pSp) framework
https://eladrich.github.io/pixel2style2pixel/
MIT License
3.2k stars 568 forks source link

Single channel input with Moco loss not working. #317

Closed floppycracken closed 1 year ago

floppycracken commented 1 year ago

Thank you for the great work. I was trying the run the code for sketch to image. The sketch is single channel, so I set the arguments input_nc and label_nc as 1, and it works, but when I try to run it with moco loss, it throws the below error. RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[4, 1, 224, 224] to have 3 channels, but got 1 channels instead

This error comes in moco_loss.py at line 42 where we feed the input to the moco model, I believe that since the model for moco loss used expects a 3-channel input that is why the error, do you have any suggestions how to handle this? Thank you

yuval-alaluf commented 1 year ago

The MoCo loss is not designed to work with single-channel inputs since it was trained on ImageNet which has three channels. What you could do is replicate the single channel to 3 channels before passing it to MoCo and this should work technically, but you many to get desirable results since the images will most likely be out-of-domain for MoCo. Hope this helps clarify the issue.

floppycracken commented 1 year ago

ah! yes that can be done but I just shifted to using 3 channel input. Thank you for the reply.