search

swyoon / normalized-autoencoders

The official repository for <Autoencoding Under Normalization Constraints> (Yoon, Noh and Park, ICML 2021).
MIT License
42 stars 18 forks source link

VAE Comparison #10

Closed GloryyrolG closed 2 years ago

GloryyrolG commented 2 years ago

Hi Sangwoong @swyoon ,

Just for curiosity, it may not be related to the main work.

Thanks for releasing scripts of 2D experiments. I saw you are using a VAE with a learnable std for comparisons. Just wondering, won't it lead to posterior collapse? Since the std can constantly be increasing to reduce the total loss. Did you use any technique to avoid this? Thanks.

https://github.com/swyoon/normalized-autoencoders/blob/27ccb74bb725768f9ba9ea6fa03a7a40867eebb1/models/modules.py#L49

vae = VAE(encoder, decoder, sigma_trainable=True, use_mean=False)
GloryyrolG commented 2 years ago

Sorry by mistake.