search

llSourcell / Pokemon_GAN

This is the code for "Generating Pokemon with a Generative Adversarial Network" by Siraj Raval on Youtube
432 stars 208 forks source link

relu vs leaky relu #13

Open miranthajayatilake opened 6 years ago

miranthajayatilake commented 6 years ago

I noted that for activation relu is used in the generator and leaky relu is used in the Discriminator. Is there a particular reason behind this?

shamanez commented 6 years ago

I think it's because discriminator tends to have vanishing gradient issues more than the generator.

Ruhan123 commented 4 years ago

Leaky ReLUs allow a small, non-zero gradient when the unit is not active.

LuposX commented 4 years ago

And why wouldn't you use leaky_relu on both. I dont see any disatvange with using leaky_Relu over relu.