Mismatch between code and paper in the gradient penalty algorithm

[bfonta]

After comparing your code and your WGAN-GP paper, there seems to be a mismatch. When you perform the gradient penalty, you do the following:

    differences = fake_data - real_data
    interpolates = real_data + (alpha*differences)
    gradients = tf.gradients(Discriminator(interpolates), [interpolates])[0]
    slopes = tf.sqrt(tf.reduce_sum(tf.square(gradients), reduction_indices=[1]))
    gradient_penalty = tf.reduce_mean((slopes-1.)**2)
    disc_cost += LAMBDA*gradient_penalty

while in the paper it seems that you are describing the following (note the differences in the first two lines):

    differences = real_data - fake_data
    interpolates = fake_data + (alpha*differences)
    gradients = tf.gradients(Discriminator(interpolates), [interpolates])[0]
    slopes = tf.sqrt(tf.reduce_sum(tf.square(gradients), reduction_indices=[1]))
    gradient_penalty = tf.reduce_mean((slopes-1.)**2)
    disc_cost += LAMBDA*gradient_penalty

You are nevertheless still sampling from a line joining fake and real data, so it should not make much difference.

[a411919924]

The 1st case: interpolates = real + alpha(fake-real) = (1-alpha)real + alphafake The 2nd case: interpolates = fake + alpha(real-fake) = alpha real + (1-alpha)fake

The two results are equal since the variable alpha is random float number between 0 and 1.

I wish this helps.