Optimiser SGD leads to KL vanishing

timbmg / Sentence-VAE

PyTorch Re-Implementation of "Generating Sentences from a Continuous Space" by Bowman et al 2015 https://arxiv.org/abs/1511.06349

580 stars 152 forks source link

Optimiser SGD leads to KL vanishing #25

Open ZhuohanX opened 3 years ago

ZhuohanX commented 3 years ago

Hi,

Thank you so much for your code, it helped me a lot.

I am wondering about optimiser, I think in the paper, they say they used SGD, but when I changed the optimiser to SGD in your code, I got the KL vanishing problem. It is okay if I use Adam, I don't know why this happens, I am wondering if you have insights about this, thank you so much!