search

igul222 / improved_wgan_training

Code for reproducing experiments in "Improved Training of Wasserstein GANs"
MIT License
2.35k stars 668 forks source link

Is the gradient penalty loss problematic when input image is large? #50

Closed shaform closed 6 years ago

shaform commented 6 years ago

Hi, Here the sum of squares is computed slopes = tf.sqrt(tf.reduce_sum(tf.square(gradients), reduction_indices=[1]))

However, when the input image is extremely large, the dimension of gradients would be huge. It seems possible that the resulting slopes would be extremely large compared to 1. Is this the case?

igul222 commented 6 years ago

In theory I agree with you; in practice I haven't found compelling evidence that it's a problem. If you find any I'd love to know :)

martinarjovsky commented 6 years ago

You might want to experiment with replacing the penalty of (norm(grad) - 1)^2 by (norm(grad) - k)^2 for a larger k > 1.

Let us know if you find anything interesting :)