search

yilundu / improved_contrastive_divergence

[ICML'21] Improved Contrastive Divergence Training of Energy Based Models
60 stars 14 forks source link

MCMC Langevin Dynamics #5

Open karimul opened 3 years ago

karimul commented 3 years ago

First of all, thanks for the paper and the source code!

I'm using this repository as a baseline code trying to reproduce the result and trying to improving MCMC sampling. You give 2 option to generate sample, either using SGLD or HMC but i didn't found the result with HMC sampling. Can you share your experience with HMC? is it better than SGLD?

yilundu commented 3 years ago

Thanks! We found that HMC was unfortunately less stable than SGLD and was not really better. Would be curious to see more exploration in this direction though!

karimul commented 3 years ago

Thank you for your response.

I have a question about how to calculate fid and IS on EBM during training, because positive examples and negative examples can be totally different depend on MCMC sampling on the same step.

yilundu commented 3 years ago

Sorry about the late response, to calculate fid and IS during training, I sampled 32 data points from the replay buffer.

karimul commented 3 years ago

Do you compare sampled from replay buffer to positive examples or negative examples?

yilundu commented 3 years ago

To positive examples