search

simplespy / SCAN-from-betaVAE

Pytorch implementation of SCAN: Learning Abstract Hierarchical Compositional Visual Concepts
21 stars 2 forks source link

question about the normalized beta #2

Open ConanCui opened 6 years ago

ConanCui commented 6 years ago

Hi You have done a good job here, but I am little confused about the normalized beta, it is shows in the code Peiyao_Sheng_beta_VAE.ipynb. The KL loss is computed by ,

KLD /= batch_size * imgSize

So I am wondering why the KL loss should be divided withthe image size? I come back to the paper and find the difinition about the normalized, in the Appendix A.6, It shows the KL loss should be computed by KL * latent size / image size, do I misunderstand?

And I think whether this influences the result or not, cause I find the beta-vae in your scan network dosen't use the normalized beta. Is this difference important？ I am looking forward your reply, thanks.

simplespy commented 6 years ago

Hi, the reason for dividing KL loss with imgSize is that the default setting of BCE Loss is mean. And as for the normalized beta, A.5 interprets the conditionally independent assumptions and because it's a hyper-parameter in experiment, I just follow the paper's setting.