search

timsainb / Tensorflow-MultiGPU-VAE-GAN

A single jupyter notebook multi gpu VAE-GAN example with latent space algebra and receptive field visualizations.
MIT License
442 stars 96 forks source link

Why you use such huge epsilon for adam, what is the philosophy behind #2

Closed menglin0320 closed 7 years ago

menglin0320 commented 7 years ago

And you use elu as the activation function, is this activation function guaranteed to be better than relu for this task?

timsainb commented 7 years ago

No philosophy behind any of this, it's just a kind of starting point if anybody wants to try to experiment around. I believe that I found that the large epsilon learned a little faster, and I think that the reason I tried it was because of ikostrikov's implementation (which has since changed significantly): https://github.com/ikostrikov/TensorFlow-VAE-GAN-DRAW