search

odegeasslbc / FastGAN-pytorch

Official implementation of the paper "Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis" in ICLR 2021
GNU General Public License v3.0
600 stars 100 forks source link

Question about loss D #28

Closed sarrbranka closed 3 years ago

sarrbranka commented 3 years ago

Why is the loss D for fake images min(0,-1-d(x_fake)) not min(0,-d(x_fake))? I feel like min(0,-1-d(x_fake)) can try to converge to -1 when the D is fed fake images. I thought the ideal output of D from real images to be 1 and from fake images to be 0. Screenshot from 2021-08-09 10-29-25

odegeasslbc commented 3 years ago

Hi. I think the statement "the ideal output of D from real images to be 1 and from fake images to be 0" is not a necessity. For a discriminator, all we hope it can do is to distinguish the real/fake images. I.e. D's output value does not need to fall into a certain range, as long as D gives real images with a score towards one direction while gives fake images a score towards an opposite direction, then it's fine. The two values can be [-1,1], can be [0,1], also can be [-inf, inf]. It simply depends on what equation you use to wrap the idea of the loss. For example, if you wrap a sigmoid function, it of course restricts the D output to [0, 1].

LearningJack commented 3 years ago

Could you help me understand the relationship with the code *err = F.relu( torch.rand_like(pred) 0.2 + 0.8 + pred).mean()** and the d loss in the paper? image

odegeasslbc commented 3 years ago
  1. Its a soft version of hinge loss. By applying randomness, the real samples are not treated as 1, but a value between 0.8 to 1. Please refer to ideas related to "label smoothing".
  2. The core idea for hinge loss is: D is no longer be optimized when if performs good enough.