search

tdeboissiere / DeepLearningImplementations

Implementation of recent Deep Learning papers
MIT License
1.81k stars 650 forks source link

Bug in mnist Gan implementation #11

Closed rajankarthiks closed 7 years ago

rajankarthiks commented 7 years ago

Hi,

I am trying to run the mnist gan code but the code is not working as intented. My Generator loss is not changing after first epoch and it remains constant at 16.0302 thereafter. Please help me. I have tried to use tensorflow backend with and without label smoothing, with and without use_mbd, also with deconv .

tdeboissiere commented 7 years ago

Play around with the learning rates (GANs are hard to train for a reason). Usually, decreasing the discriminator learning rate is enough.

I got decent results with:

    python main.py --backend tensorflow --generator deconv --use_mbd
    opt_dcgan = Adam(lr=1E-3, beta_1=0.5, beta_2=0.999, epsilon=1e-08)
    opt_discriminator = SGD(lr=1E-3, momentum=0.9, nesterov=True)
rajankarthiks commented 7 years ago

Thank you so much @tdeboissiere .. I tried a smaller learning rate of 1e-4 and it work good. But still I am wondering why i was not able to reproduce the it. weird as always..