I find an implementation detail of Importance Weighted Autoencoder concerning:
First, it is SGVB 1 (Auto-encoding Variational Bayes Eq 6) should be used instead of SGVB 2 (Auto-encoding Variational Bayes Eq 7, 10) for estimating the evidence lower bound
Second, the importance weights should be detached, or the gradient is different from (Importance Weighted Autoencoder Eq 14). To be specific:
in EQ 14: grad = \Sum w * \nabla ELBO
in current impl, grad = \Sum \nabla (w ELBO) = \Sum w \nabla ELBO + \nabla w * EBLO, an extra term is introduced
I made a loosy PR as I do not know how to run this repo properly. I would appreciate any effort to review and correct it.
Hi thanks for this amazing project, I like it.
I find an implementation detail of Importance Weighted Autoencoder concerning:
I made a loosy PR as I do not know how to run this repo properly. I would appreciate any effort to review and correct it.
Thanks