search

jaanli / variational-autoencoder

Variational autoencoder implemented in tensorflow and pytorch (including inverse autoregressive flow)
https://jaan.io/what-is-variational-autoencoder-vae-tutorial/
MIT License
1.15k stars 256 forks source link

the expected_log_likelihood is not a expected value, but only an log likelihood #20

Closed jiongjiongli closed 5 years ago

jiongjiongli commented 5 years ago

May I confirm line 133 in train_variational_autoencoder_tensorflow.py:

 expected_log_likelihood = tf.reduce_sum(p_x_given_z.log_prob(x),
                                          [1, 2, 3])

It seems the reduce_sum returns not an expected value, but only a log likelihood.

jaanli commented 5 years ago

Good question—it is an expectation (a Monte Carlo estimate), but using a single sample. The expectation is over q_z, as q_z.sample() only draws a single sample: https://github.com/altosaar/variational-autoencoder/blob/96337fa367e720bd59da6979b64b2a526e96cfc1/train_variational_autoencoder_tensorflow.py#L104