votingworks / arlo

GNU Affero General Public License v3.0
141 stars 25 forks source link

Update audit report to correctly display RCV data #1800

Open umbernhard opened 1 year ago

umbernhard commented 1 year ago

Currently our audit reports and data validation logic assume ballots contain binary digits for selections, so we should update it to account for ranks. Specifically, in a ballot polling audit report, two sections will need updating. First is the rounds section. Currently, it looks like this:

Round Number Contest Name Targeted? Sample Size Risk Limit Met? P-Value Start Time End Time Audited Votes    
1 Pizza Targeted 53 No 0.2256268681 2021-10-29T18:03:52.591488+00:00 2021-11-09T17:48:40.057251+00:00 Pineapple: 30 Never pineapple: 23 Ballots not found (counted for loser): 0

In a ranked-choice context, we won't have a tally at the end of the section like this, since it doesn't make sense to tally ranked ballots. A more appropriate thing would be to list the assertions we're testing against:

Round Number Contest Name Targeted? Sample Size Risk Limit Met? P-Value Start Time End Time Assertions    
1 Pizza Targeted 53 No 0.2256268681 2021-10-29T18:03:52.591488+00:00 2021-11-09T17:48:40.057251+00:00 Never pineaapple eliminated before Pineapple Pineapple not elminated next in round 2

Additionally, the ballot-by-ballot reporting will need updating, to look more like it does in ballot comparison audits. What we have now:

Jurisdiction Name Batch Name Ballot Position Ticket Numbers: Pizza Audited? Audit Result: Pizza
Belknap County 3 2110 Round 1: 0.000336016626577407 AUDITED Pineapple

What we will need:

Jurisdiction Name Batch Name Ballot Position Ticket Numbers: Pizza Audited? Audit Result: Pizza
Belknap County 3 2110 Round 1: 0.000336016626577407 AUDITED 1: Pineapple, 2: Not Pineapple
umbernhard commented 1 year ago

Alternatively, for the rounds section, we could do one row per assertion, since each assertion gets its own p-value. That would make it easier for others to reproduce our results.