SGD +momentum + noise "=" SGHMC - Githubissues

andrewgordonwilson / bayesgan

Tensorflow code for the Bayesian GAN (https://arxiv.org/abs/1705.09558) (NIPS 2017)

Other

1.02k stars 176 forks source link

SGD +momentum + noise "=" SGHMC #18

Open emiled16 opened 3 years ago

emiled16 commented 3 years ago

Hello, I was going through your paper and through the paper of SGHMC and I understand that :

SGD + noise equates to SGLD
SGD + momentum + noise equates to SGHMC However, I don't understand what Adam + noise which is what you used in your code equates to? in the paper you said that you were going to use SGHMC. Is it reasonable to assume that Adam + noise is also equivalent to SGHMC? if so, can you please say why?

Thanks