CDCgov / multisignal-epi-inference

Python package for statistical inference and forecast of epi models using multiple signals
https://cdcgov.github.io/multisignal-epi-inference/
9 stars 1 forks source link

Create methods for simulating from prior and computing predictive distributions #218

Open damonbayer opened 4 days ago

damonbayer commented 4 days ago

Adds posterior_predictive() and prior_predictive() methods to model class. See model/docs/example_with_datasets.qmd for a demonstration of the posterior_predictive function. I don't love the interface, but I think it's the best we can do, based on how numpyro works. I'm also not sure the best way to handle the documentation, since they are just thin wrappers around numpyro functions.

The visualizations leave a lot to be desired. I'm hoping @sbidari could maybe work on them as our resident ArviZ expert. In particular, I think the use of Highest posterior density intervals make the results look worse than they are. As far as I can tell, there is no way to make a plot with equal tail intervals in ArviZ, which would be surprising, if true.

I kind of think this PR could just be creating the methods and a separate PR could be dedicated to visualizations.

Visualization to be addressed in https://github.com/CDCgov/multisignal-epi-inference/issues/221.

Closes https://github.com/CDCgov/multisignal-epi-inference/issues/202.

codecov[bot] commented 4 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 92.59%. Comparing base (84f7809) to head (3341103). Report is 3 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #218 +/- ## ========================================== + Coverage 92.47% 92.59% +0.11% ========================================== Files 40 40 Lines 877 891 +14 ========================================== + Hits 811 825 +14 Misses 66 66 ``` | [Flag](https://app.codecov.io/gh/CDCgov/multisignal-epi-inference/pull/218/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CDCgov) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/CDCgov/multisignal-epi-inference/pull/218/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CDCgov) | `92.59% <100.00%> (+0.11%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CDCgov#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

sbidari commented 3 days ago

The visualizations leave a lot to be desired. I'm hoping @sbidari could maybe work on them as our resident ArviZ expert. In particular, I think the use of Highest posterior density intervals make the results look worse than they are. As far as I can tell, there is no way to make a plot with equal tail intervals in ArviZ, which would be surprising, if true.

I don't think there is a built in eti_plot feature in ArviZ. We can obtain equal tail intervals using stat_focus="median" (see here). Passing these eti intervals in place of hdi intervals in plot_hdi should work although it's a bit hacky solution

seabbs commented 2 days ago

Attention: Patch coverage is 28.57143% with 10 lines in your changes missing coverage. Please review.

Being annoying but unless a new issue is made this should probably be addressed here. From a skim it looks mostly like exception handling.

Very exciting to get this functionality in place!

damonbayer commented 2 days ago

@seabbs Sorry, this PR is a bit of a mess. Converting to draft to indicate that is not ready for a full review. Just wanted to get some early feedback.

seabbs commented 2 days ago

No worries at all! I'm just being an annoying GitHub 👻 today!