search

ecohealthalliance / open-rvfcast

Wellcome Open RVFCast project repository
Other
0 stars 2 forks source link

Model reporting for iterative comparisons #5

Open emmamendelsohn opened 1 year ago

emmamendelsohn commented 1 year ago 
### Tasks
- [ ] https://github.com/ecohealthalliance/open-rvfcast/issues/69
- [ ] Model diagnostics and interpretation (e.g., test data performance, mcmc chains, variable importance)
- [ ] Real time predictions (e.g., maps, figures/maps showing which variables are most contributing to predictions, rolling look at predictions and actual conditions since model was fit). Example map showing relative contribution of variables by pixel: https://www.mdpi.com/2078-1547/13/2/66#B52-challenges-13-00066
- [ ] Model accuracy metrics
emmamendelsohn commented 2 months ago

In general, this should be a report that is generated along with the first model version to enable comparison as you iterate on the model.

emmamendelsohn commented 2 months ago

This might be helpful from REPEL where we used relic to compare model updates: https://github.com/ecohealthalliance/repel2/blob/dev/reports/repel_model_iterative_comparison_checks.Rmd. The model statistics are different from REPEL (lme4) but the comparison framework may be useful.