Closed cjf00000 closed 6 years ago
This would indeed make sense to me, but when I tried it I was unable to find an experimental setup where doing that improved the results in any consistent way. In my experiments it at least doesn't seem to hurt though, so if you want more theoretical elegance and don't mind the extra computation time you can probably do that and it'll still work.
It seems that the gradient penalty term also has non-zero gradient w.r.t. the generator parameters. Shouldn't the penalty term be added to both the discriminator and the generator objective to ensure that the mini-max procedure finds a saddle point?