Loss Curve & Model Size

[kingnobro]

Hi Ming Tao, thanks for your code. It's really helpful for me. When I read the code and run the experiment, I have two questions.

1. I add the following codes in the train() function in modules.py to get more information about the loss during training:

   if get_rank() == 0:
   writer.add_scalar('Loss/errD_real', errD_real.item(), base_step + step)
   writer.add_scalar('Loss/errD_fake', errD_fake.item(), base_step + step)
   writer.add_scalar('Loss/errD_mis', errD_mis.item(), base_step + step)
   writer.add_scalar('Loss/errD_MAGP', errD_MAGP.item(), base_step + step)
   writer.add_scalar('Loss/errD', errD.item(), base_step + step)
   writer.add_scalar('Loss/errG', errG.item(), base_step + step)

   However, when I train the CUB Birds dataset, the loss curves seem to oscillate continuously. Did you observe the same phenomenon and know the reason?

2. In the models/GALIP.py, it seems that the number of layers of the Generator relies on the parameter ngf. But in the configs, birds.yml and coco.yml has the same value, where nf=64. So you mean this nfg is enough for different datasets, even for COCO12M? Because I want to train a GAN with more capacity later.

Again, very useful code!

[tobran]

Hi, the adversarial learning is dynamic and it is typical for the adversarial loss curve to oscillate continuously. The larger GALIP can reach better performance, it is more obvious on larger complex datasets. The GALIP(ngf=128) of COCO can reach FID ~4.7.