search

junyanz / CycleGAN

Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.
Other
12.22k stars 1.94k forks source link

Style couldn't transfer using horse2zebra dataset #135

Closed Kaede93 closed 3 years ago

Kaede93 commented 3 years ago

Hello, I tried to train a cyclegan for horse2zebra transfer, and I found it's really hard to get the results.

  1. I wrote the algorithm with Keras because my PC have some problem with using pytorch.

  2. For the training data, I picked all images in trainA (1067 images) and same number in trainB. Without any augmentation and get 1 image in both dataset randomly.

  3. Using Unet128 for generators, patchGAN for discriminators. Adam with learning rate 4e-4 for generators and 2e-2 for discriminators, lr decrease 50% per 50 epochs. lambda_cyc = 5, lambda_id = 0.25 (results of 5 * 0.5)

  4. After 200 epochs training (over 200K iterations, which is your recommendation reply in other issue), I found almost all pictures just have little changes. For example, the horse with some stripes on its body while the head without any changes, and the zebra still keep its stripes and just get little brown on its skin. In fact, I can get the same results in the first 50 epochs.

after 50 epochs, [X] d_loss = 0.0346/100%, lsgan = 0.6427, cyc = 0.0591, id = 0.0384; [Y] d_loss = 0.0834/100%, lsgan = 0.9608, cyc = 0.1087, id = 0.0534 after 200 epochs, [X] d_loss = 0.0101/100%, lsgan = 0.2967, cyc = 0.0298, id = 0.0263; [Y] d_loss = 0.0925/95.92%, lsgan = 0.4896, cyc = 0.0916, id = 0.0271

I think the model was converged after 50 epochs because the discriminators were too strong, can you give me some advice? Will label smoothing work in this situation?

  1. Overfitting problem (?) also seem in my results. For example, I found same patterns of stripe putting on the horse in the training dataset, many horses have same stripe. I considered this as an overfitting problem. Should I apply augmentation on the training datasets?

  2. What's the correct changes while training the horse2zebra, can you give me some tips?

Thank you for you time, have a good day.

junyanz commented 3 years ago

I recommend that you follow the network architectures and hyper-parameters of the original paper before making further changes. We use the same learning rate for G and D. Your discriminator's learning rate might be too big. We also use resnet_6blocks or resnet_9blocks instead of Unet128. Unet doesn't work very well for the CycleGAN setting.

There are two Keras implementations. You can have a look at theirs. https://github.com/tjwei/GANotebooks https://github.com/simontomaskarlsson/CycleGAN-Keras

Kaede93 commented 3 years ago

Thank you for your reply and I getting more reasonable results after modified my network according your recommendations. And I'm wondering that Why unet doesn't works very well compared with resnet? Is that an experimental observation? In my opinion, the unet will generate images with higher quality because it has more skip connections than resnet 6 or 9 blocks. Please correct me if I'm wrong. Thank you so much for your time again!

junyanz commented 3 years ago

It is an experimental observation. The resnet also has many residual blocks.

Kaede93 commented 3 years ago

Thank you! Have a nice day!