search

soumith / ganhacks

starter from "How to Train a GAN?" at NIPS2016
11.43k stars 1.66k forks source link

> @KrnTneja : Thanks for your tricks. Could you provide any code to do it? I also meet the problem of loss D goes to zero #67

Closed shaurov2253 closed 4 years ago

shaurov2253 commented 4 years ago

@KrnTneja : Thanks for your tricks. Could you provide any code to do it? I also meet the problem of loss D goes to zero

There isn't really any code to show. Just ensure that last layer of your discriminator is not a sigmoid layer i.e. output shouldn't be constrained to [0,1]. I was using PyTorch, where I had to use torch.nn.BCEWithLogitsLoss instead of torch.nn.BCELoss.

Originally posted by @KrnTneja in https://github.com/soumith/ganhacks/issues/36#issuecomment-493886559

shaurov2253 commented 4 years ago

I understand that getting rid of sigmoid function resolved this problem (using torch.nn.BCEloss instead of torch.nn.BCEwithLogitsLoss). I have two questions:

  1. Did you use any other activation function instead of 'sigmoid' or just didn't use any activation for the last layer?
  2. Can you please share any insight about why this worked?