feat: add snaphot tests of canned forecasters

dshemetov commented 3 months ago

Checklist

Please:

[x] Make sure this PR is against "dev", not "main".
[x] Request a review from one of the current epipredict main reviewers: dajmcdon.
[ ] ~~Makes sure to bump the version number in DESCRIPTION and NEWS.md. Always increment the patch version number (the third number), unless you are making a release PR from dev to main, in which case increment the minor version number (the second number).~~
[ ] ~~Describe changes made in NEWS.md, making sure breaking changes (backwards-incompatible changes to the documented interface) are noted. Collect the changes under the next release number (e.g. if you are on 0.7.2, then write your changes under the 0.8 heading).~~

Change explanations for reviewer

Adds snapshot tests. Could add more snapshots with different parameter settings, but I went with a few for now.

Magic GitHub syntax to mark associated Issue(s) as resolved when this is merged into the default branch

Resolves #306

dshemetov commented 3 months ago

Ran into an issue where the CI and my local snapshots disagreed. Ran renv::update() locally, updated packages, and it seemed to fix it by updating the snapshots. A little concerning that our canned forecasts can change in the 3rd decimal place based on updated dependencies. But at least these snapshots will help us identify when such changes come from our work or from an upstream dependency.

dshemetov commented 2 months ago

So for each of our canned forecasters, I included the forecast outputs of a few (~default) forecaster (using included JHU data) as snapshots in the tests. Now these tests will compare the output of those forecasts on developer branches and will give us (at least some) confidence we have not altered these forecaster outputs with our code changes. And if the forecasts have changed, you can run testthat::snapshot_review() to get a nice diff interface, where you can decide whether to accept or reject the new snapshots.

dajmcdon commented 2 months ago

Cool! And it runs on R CMD Check?

dshemetov commented 2 months ago

Yup, it's part of the tests, so it runs with devtools::test() too.

cmu-delphi / epipredict