Audit planner

[benadida]

Problem

Right now, in order to help states plan their audits, VotingWorks support runs audit scenarios by creating hypothetical audits and seeing where the numbers land. One of the main questions they're trying to answer is which audit type to use (since different audit types will require different amounts of work).

There are sample size calculators outside of Arlo that can be used for ballot polling and ballot comparison, but not for batch comparison.

Proposed Solution

Create an audit planner tool to compare different audit types without having to go through the whole flow to create an audit.

Requirements

• Takes as input candidate vote totals, total ballots cast, and number of winners
• Estimate sample sizes for ballot polling, ballot comparison, and batch comparison
• Allow trying out different risk limits
• Only needs to estimate sample size for one contest at a time
• Sample size estimates should be shareable alongside the input data
• Doesn't require logging into Arlo (so it can be used broadly)

Sample size to generate:

• Ballot polling: 90% probability of completion (see audit_math/bravo.py)
• Ballot comparison: standard (see audit_math/supersimple.py)
• Batch comparison: estimate given by this formula: - ln(risk limit) / diluted margin + 2, where risk limit is a decimal and diluted margin = (winner - loser) / total ballots cast for the smallest winner - loser margin among all the candidate pairs.

Note that the unit for batch comparison is batches, not ballots.

[ginvdr]

For this calculator, total ballots, number of winners, and candidate vote totals will be useful. It would be very helpful if it's built so that information is entered and a user can then see sample sizes for all audit types without re-entry.

[jonahkagan]

Mockups

[arsalansufi]

Follow-up issues:

• https://github.com/votingworks/arlo/issues/1601
• https://github.com/votingworks/arlo/issues/1603
• https://github.com/votingworks/arlo/issues/1605
• https://github.com/votingworks/arlo/issues/1609
• https://github.com/votingworks/arlo/issues/1610
• https://github.com/votingworks/arlo/issues/1611
• https://github.com/votingworks/arlo/issues/1612
• https://github.com/votingworks/arlo/issues/1613
• https://github.com/votingworks/arlo/issues/1614

[arsalansufi]

Now that I've opened issues for follow-ups (above), I'm going to mark this base issue as done! We can prioritize the follow-ups as we see fit, many of which are prerequisites for a public Twitter post