search

alarm-redist / redist

Simulation methods for legislative redistricting.
https://alarm-redist.github.io/redist/
GNU General Public License v2.0
66 stars 23 forks source link

Diagnostic Check Function #95

Closed christopherkenny closed 2 years ago

christopherkenny commented 2 years ago

We have a request to make some diagnostic check functions that can be run with a check_... function with guided interpretation. This brings back Kosuke's suggestion for a set of diagnostic plots (rather than us manually running them each time) from April, before we got side-tracked with the Census project.

We would want tests for each of the major algorithms: [ ] SMC [ ] Flip [ ] Merge-split

cc: @bfifield -- This would likely supersede the old flip diagnostic functions.

kosukeimai commented 2 years ago

For Flip and Merge-split, one diagnostic is based on multiple chains. For example, we could run a vanilla SMC and get starting maps.

bfifield commented 2 years ago

One thing I've loved in the sensemakr package are the verbal descriptions of how to interpret the diagnostic measures (see after summary(sensitivity), at https://github.com/carloscinelli/sensemakr#basic-usage). I think it would really help practitioners with less statistical training to have something similar as this is being built out, so that they can interpret the diagnostics more easily and do tuning well.

CoryMcCartan commented 2 years ago

Plan for this: version 4.1, implementing a new summary() generic that spits out some diagnostics based on MCMC/SMC