search

ericjang / gumbel-softmax

categorical variational autoencoder using the Gumbel-Softmax estimator
MIT License
425 stars 101 forks source link

Question regarding KL calculation #7

Closed backpropper closed 6 years ago

backpropper commented 6 years ago

In the gumbel_softmax_vae_v2 notebook, for calculating the KL in case of Relaxed Prior, shouldn't the KL be y * (log q_y - log p_y). So the product with y is missing?

backpropper commented 6 years ago

I see that its already a part the log_prob function.

Diego999 commented 5 years ago

@backpropper could you please explain what we don't have to multiply do p_y.log_prob(y).exp() * (q_y.log_prob(y) - p_y.log_prob(y)) ?