search

nmichlo / disent

🧶 Modular VAE disentanglement framework for python built with PyTorch Lightning ▸ Including metrics and datasets ▸ With strongly supervised, weakly supervised and unsupervised methods ▸ Easily configured and run with Hydra config ▸ Inspired by disentanglement_lib
https://disent.michlo.dev
MIT License
122 stars 18 forks source link

Make beta tc loss more stable using torch.logsumexp #41

Closed meffmadd closed 1 year ago

meffmadd commented 1 year ago

The code is from https://github.com/YannDubs/disentangling-vae The PyTorch implementation uses a more stable implementation than just chaining .log().sum.exp()

meffmadd commented 1 year ago

Will also look into using torch.Normal for _betatc_compute_gaussian_log_density.

codecov[bot] commented 1 year ago

Codecov Report

Base: 70.04% // Head: 70.04% // No change to project coverage :thumbsup:

Coverage data is based on head (1446fd8) compared to base (de159f2). Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #41 +/- ## ======================================= Coverage 70.04% 70.04% ======================================= Files 135 135 Lines 7538 7538 ======================================= Hits 5280 5280 Misses 2258 2258 ``` | [Impacted Files](https://codecov.io/gh/nmichlo/disent/pull/41?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Nathan) | Coverage Δ | | |---|---|---| | [disent/frameworks/vae/\_unsupervised\_\_betatcvae.py](https://codecov.io/gh/nmichlo/disent/pull/41/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Nathan#diff-ZGlzZW50L2ZyYW1ld29ya3MvdmFlL191bnN1cGVydmlzZWRfX2JldGF0Y3ZhZS5weQ==) | `100.00% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Nathan). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Nathan)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

nmichlo commented 1 year ago

Sorry, for the delay getting back to you about this PR.

Good catch! Thank you so much for this contribution!

EDIT: I notice the old contributions are still part of the history, if there is anything in future it might be easier to start the commits from the main branch after synchronizing with upstream changes 😁