Batch norm in WGAN discriminator

cyclomon / 3dbraingen

Official Pytorch Implementation of "Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Network" (accepted by MICCAI 2019)

MIT License

126 stars 38 forks source link

Batch norm in WGAN discriminator #6

Open KhrystynaFaryna opened 4 years ago

KhrystynaFaryna commented 4 years ago

Hi, I was wondering why did you guys decide to use batch norm in a critic of WGAN-GP. The paper on Improved training of WGAN(the one where gradient penalty is proposed) advises against it. Thanks in advance!

cyclomon commented 4 years ago

Hi, Although we noticed that it is not a good choice of using BN in discriminator of WGAN-GP, we found that using batchnorm stabilized training. We also experimented with Instance Norm , LayerNorm, and without normalization, but we could not find a better model. Now we moved to another project so we still don't know the exact reason why this works. Maybe further experiments and analysis will be needed to figure it out. Thanks!