search

tdeboissiere / DeepLearningImplementations

Implementation of recent Deep Learning papers
MIT License
1.81k stars 652 forks source link

WGAN-GP update formula wrong? #67

Open Imperssonator opened 6 years ago

Imperssonator commented 6 years ago

Line 76 of train_wgan_GP: X_hat = X_real + epsilon * (X_fake - X_real)

From the paper this should be (Algorithm 1 Line 6): X_hat = epsilon X_real + ( 1 - epsilon ) X_fake

In the format you used above: X_hat = X_fake + epsilon * (X_real - X_fake)

In other words, X_hat should be a small nudging of X_fake toward X_real, not a small nudging of X_real toward X_fake...

Let me know if I'm missing something or if there's a reason it was done this way.

Imperssonator commented 6 years ago

Looks like somebody else raised this issue before and it was closed, but the problem is still there.

sarthmit commented 6 years ago

@Imperssonator I think both the lines are equivalent. Notice that since epsilon is drawn from a uniform distribution, you can equivalently replace epsilon by (1-epsilon), both of these expressions would be equivalent. Now note that if you replace epsilon by (1-epsilon) in the paper's definition, you would get the one in the code. So both are correct.