search

YongfeiYan / Gumbel_Softmax_VAE

PyTorch implementation of a Variational Autoencoder with Gumbel-Softmax Distribution
202 stars 37 forks source link

Some questions on KLD #1

Closed gaosh closed 6 years ago

gaosh commented 6 years ago

In your code the KL divergence is calculated by: KLD = torch.sum(qy * (log_qy - 1. / categorical_dim), dim=-1).mean() I think, for the 1. / categorical_dim, it should be replaced by the torch.log(1. / categorical_dim), otherwise, it is not the KL divergence.

YongfeiYan commented 6 years ago

Thanks gaosh, I had that bug fixed.