search

votingworks / arlo

GNU Affero General Public License v3.0
143 stars 26 forks source link

Fix round 2+ stuck jurisdictions #1679

Open jonahkagan opened 2 years ago

jonahkagan commented 2 years ago

https://votingworks.slack.com/archives/CKCVA0F9S/p1667525834769069

carolinemodic commented 2 years ago

I have recreated the buggy state in the Batch Comparison audit in "Vx Internal - Caroline's Testing Organization" election ID 327e3af4-2912-4c23-92b5-0dd57b441f22

jonahkagan commented 2 years ago

There are a few intersecting things going on here:

  1. There are 18 total batches in this audit. The round 1 sample size was 17 and the round 2 sample size was 5, so adding those up gives a total sample size of 22, which is greater than 18, so that's why there's a warning that a full hand tally is required.
  2. In round 1, even though 17 samples were drawn, only 11 batches were covered by those samples. The 5 batches drawn in round 2 only covered 2 new batches. That totals 13 batches. So Arlo didn't set you up to do a full hand tally even though it's letting you know that you do in fact need one. This is one of the cases we need to cover when we tackle #1525.
  3. Acton had 3 batches sampled in round 2, all of which were audited in round 1 already. Acton shows up as "in progress" because they haven't finalized their results, but they can't finalize their results because their interface says they have no batches sampled (which is true enough, they have no new batches sampled). This feels like a bug.

To address this bug, there are a few things to consider:

Perhaps the jurisdiction should be auto-finalized in this case, or not required to finalize.

This also makes me realize there's another bug lurking behind this: If we draw a sample consisting entirely of batches (or ballots) that were audited in previous rounds, things will be similarly stuck. It's not clear to me what should happen in this case.

jonahkagan commented 2 years ago

A workaround is to finalize the impacted jurisdiction in the db using a script.

arsalansufi commented 2 years ago

Took care of the batch audit bug where Arlo indicates in a subsequent round that a full hand tally is needed but doesn't select all remaining unaudited batches in https://github.com/votingworks/arlo/pull/1703. For this specific case, we'll also no longer run into the stuck jurisdictions problem because we intentionally don't reselect already audited batches.

For non-full-hand-tally round 2+'s, the stuck jurisdictions issue remains a problem.