search

junyanz / pytorch-CycleGAN-and-pix2pix

Image-to-Image Translation in PyTorch
Other
22.42k stars 6.23k forks source link

Cheating GAN? #1184

Open 3DTOPO opened 3 years ago

3DTOPO commented 3 years ago

I am using the CycleGan model and it works perfectly for most images, but it sometimes creates garbage like the generator is cheating.

epoch039_fake_B_rgb

If I lower the learning rate enough, the problem goes away, but the effect it produces is too weak to be useful. I've also tried lowering the Adam momentum, but the issue still occurs. I've tried using lsgan, vanilla, and disabling gan loss. If I disable gan loss, the "garbage" issue goes away, but the effect is way too weak. I've lowered the learning rate to 0.00001 and ran it for 200 epochs but the effect was too weak. I've tried setting history to 500.

If I increase the number of generator and discriminator filters the issue is less severe, but I am trying to use as small model as possible to get the results I need (the generator needs to run on mobile).

Any suggestions? The training set is 41,000 images. Do I need more images? Do I need to use a very low learning rate and let it run for much longer (eg 500 epochs at 0.00001)?

Each setting I try takes days (on the GeForce 3090), so suggestions would be very much appreciated.

3DTOPO commented 3 years ago

Here's another example of "garbage". epoch007_fake_B_rgb copy

yipengliu1993 commented 3 years ago

Hi, just wondering are you using ubuntu or windows when running cycle gan? My 3090 is not working at all

3DTOPO commented 3 years ago

Ubuntu.

junyanz commented 2 years ago

It looks like that the generator is hiding information for the reconstruction purpose. See this study for more details. It is hard to know what's going on without seeing the entire dataset/task and trying multiple models. If you haven't tried it, you could train a model on smaller patches rather than the entire image.