maho3 / ltu-ili

Robust ML in Astro
https://ltu-ili.readthedocs.io/en/latest/
46 stars 9 forks source link

Adds Evidence estimator module #157

Open maho3 opened 6 months ago

maho3 commented 6 months ago

Adds implementations of evidence estimators from:

Implements runners for each of these methods, and compares their ability to measure $\log K$ in notebooks/evidence.ipynb.

Still to-do:

codecov-commenter commented 6 months ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 0.46948% with 212 lines in your changes missing coverage. Please review.

Project coverage is 85.88%. Comparing base (3019bbd) to head (4733d3b).

Files Patch % Lines
ili/evidence/runner.py 0.00% 144 Missing :warning:
ili/evidence/utils.py 0.00% 67 Missing :warning:
ili/evidence/__init__.py 0.00% 1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #157 +/- ## =========================================== - Coverage 96.95% 85.88% -11.08% =========================================== Files 23 26 +3 Lines 1644 1856 +212 =========================================== Hits 1594 1594 - Misses 50 262 +212 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

DeaglanBartlett commented 5 months ago

As highlighted in https://github.com/astro-informatics/harmonic/issues/271, using the harmonic code for models with only one parameter does not work, since the normalising flow expects there to be at least two parameters. Even if this is not fixed in harmonic, we should create a work around here. Creating a simple normalising flow which we then wrap to become a hm.model.FlowModel object so that it has the correct functionality should be sufficient. This edge case should be explicitly included in the unit tests.