search

ht014 / BGAN

41 stars 22 forks source link

What does the KL_loss mean? #4

Open zlyx525 opened 6 years ago

zlyx525 commented 6 years ago

Thanks very much for sharing the code and I think this is very great job. But I have some questions in understanding the code. Firstly, what does the KL_loss mean? According to my understand, the SSE_loss in the code corresponds to the MSE_loss in the paper, but you didn't mentioned the KL_loss in the paper? Secondly, you did't use the KL_loss and SSE_loss when updating the params, why? Thirdly, z_x = tf.add(z_x_mean, tf.multiply(tf.sqrt(tf.exp(z_x_log_sigma_sq)), eps)) # grab our actual z, what does "grab actual z "mean? Finally, What's the role of the function "average_gradients" @htconquer

zoeyuchao commented 5 years ago

Hi, I have the same questions as yours. Have you find the answers?

ozen commented 5 years ago

I came across this during research.

What was implemented is a variational autoencoder (VAE). So the output of the encoder is a probability distribution instead of a latent vector. Learning more about VAEs would let you answer your own questions.

I suggest this blog post: https://lilianweng.github.io/lil-log/2018/08/12/from-autoencoder-to-beta-vae.html

"Loss Function" section explains the KL divergence etc.

"Reparameterization Trick" section explains "grabbing actual z".

I haven't read the paper fully yet, haven't the authors mentioned that this is a VAE?