Open miranthajayatilake opened 6 years ago
I think it's because discriminator tends to have vanishing gradient issues more than the generator.
Leaky ReLUs allow a small, non-zero gradient when the unit is not active.
And why wouldn't you use leaky_relu on both. I dont see any disatvange with using leaky_Relu over relu.
I noted that for activation relu is used in the generator and leaky relu is used in the Discriminator. Is there a particular reason behind this?