kodalinaveen3 / DRAGAN

A stable algorithm for GAN training
MIT License
239 stars 35 forks source link

Difference between paper and code in obtaining perturbed inputs #6

Closed Netzeband closed 6 years ago

Netzeband commented 6 years ago

In your paper (page 6. equation 1), you calculate the gradient penalty in this way:

grafik

So it looks like you simply add pixelwise Gaussian Noise with mean 0 and standard-deviation c to the inputs of the discriminator. Furthermore you suggest to use c = 10

However, in your code, you you calculate the perturbed inputs in this way:

def get_perturbed_batch(minibatch):
    return minibatch + 0.5 * minibatch.std() * np.random.random(minibatch.shape)

So here you add pixelwise uniform noise from the interval [0.0, 0.5 * minibatch.std() ), which is very different to the formulation of noise in the paper. Furthermore your code still contains the interpolation issue from https://github.com/kodalinaveen3/DRAGAN/issues/5

Of course you wrote "We use small pixel-level noise but it is possible to find better ways of imposing this penalty. However, this exploration is beyond the scope of our paper." But maybe there are important reasons why you use this type of X_p generation in the code?

kodalinaveen3 commented 6 years ago

The code is really old, and we updated our paper with a simple clean version of the penalty later. If you notice the old one, we were basically setting 'c' automatically based on std() of the pixel values. In the new one, we leave them as hyperparameters