search

broadinstitute / CellBender

CellBender is a software package for eliminating technical artifacts from high-throughput single-cell RNA sequencing (scRNA-seq) data.
https://cellbender.rtfd.io
BSD 3-Clause "New" or "Revised" License
297 stars 54 forks source link

Don't crash with ZeroDivisionError if initial test ELBO is the best test ELBO #312

Closed alecw closed 7 months ago

alecw commented 11 months ago

The --final-elbo-fail-fraction test checks that the final test ELBO isn't much worse than the best test ELBO. It checks the (distance between final test ELBO and best test ELBO)/(distance between initial test ELBO and best test ELBO).

This causes a ZeroDivisionError if best test ELBO == initial test ELBO.

Solution: If --final-elbo-fail-fraction is enabled && final test ELBO < best test ELBO && initial test ELBO == best test ELBO then training is considered to have failed.

sjfleming commented 7 months ago

Hi @alecw , any chance you could rebase this on top of dev post #345 ? I think the test_elbo referred to here should now probably be model.loss['test']['elbo']

alecw commented 7 months ago

Hi Stephen, yes, fixed conflicts, addressed removed local variable and rebased. thanks!

sjfleming commented 7 months ago

Thanks so much!