search

FreeAndFair / ColoradoRLA

Software to facilitate risk-limiting audits at the state level, developed for the state of Colorado.
Other
20 stars 8 forks source link

Specify minimum set of tables necessary for export for public Audit Center #862

Open sfsinger19103 opened 6 years ago

sfsinger19103 commented 6 years ago

Here's a minimum protocol which pretty much determines the minimum set of tables.

source: https://docs.google.com/document/d/1olo2W38kMi4mUH1S3UxRfFMlrszXHjLNK_dLuAMLT1w/edit#heading=h.ra35frr05gsz

Minimum protocol to check correctness of Colorado’s Risk-Limiting Tabulation Audit If an independent entity wishes to check the correctness of the Colorado risk-limiting tabulation audit, it is sufficient for that entity to carry out the following steps. We believe these to be a minimum set of necessary items to verify that the audit was correctly conducted.

Note that this protocol does not address certain aspects of public verification of the election results, but only the correct performance of the Risk Limiting Audit from the ballot cards in custody of the Counties and the ballot manifest and CVR files exported from the voting system. In particular, chain of custody issues for the ballot cards before tabulation are not addressed by this protocol.

There is no break in the chain of ballot card custody between the tabulation of the results by the system under audit and the Audit Board review of ballot cards. Observation: details of observation depend on chain of custody protocol. Tabulating contests “selected for audit” on the CVRs used for the audit yield the announced tabulated results. Requires: Data: CVR files Data: List of contests selected by Secretary of State for audit Data: Announced tabulated results for contests selected for audit Calculation: tabulate results in contests selected for audit from CVR entries, compare to announced results. These should match. The set of ballot cards listed in each ballot manifest exactly matches the set of ballot cards represented in the corresponding CVR file. Requires: Data: The ballot manifest files Data: The CVR files Calculation: match the data lines in the CVR file one-to-one with the cards listed in the corresponding ballot manifest file The list of ballot cards selected for audit has been determined after the finalization of the manifests and CVR files. Requires: Data: Hashes of CVR and ballot manifest files (released to public before selection of random seed) Data: full CVR and ballot manifest files (can be released later) Calculation: calculate hashes of the full files and compare to the hashes released before the selection of the random seed. These must match. Random sequence of ballot cards has been properly selected according to the specified random algorithm. Requires: Observation: Random seed generated at public meeting Data: Random sequence of ballot cards used for the audit. (This random sequence may include duplicates.) Data: Ballot manifest file Data: CVR file Calculation: Use ballot manifest file to number ballot cards represented in the ballot manifest sequentially. Then calculate the pseudo-random sequence via prescribed algorithm based on the random seed. Together these determine the random sequence of ballot cards. This should match the sequence used in the RLA. The list of ballots selected for audit matches the list actually reviewed by the Audit Board. Data: CVR file Calculation: match lines of the CVR file to the ballots listed in the ballot manifest. Data: List of ballot cards assigned for review by Audit Board. This list can be created from the random sequence by removing duplicates. Observation: physical assembly of ballot cards for Audit Board review Audit Board interpretation matches content of paper ballot cards Observation: Audit Board review of paper ballot cards Audit Board intent is reflected in the computer record of Audit Board intent. Observation: Audit Board review of ballot cards Data: computer record of Audit Board review Calculation: the records from the observations should match the computer record. Risk limit achieved (or hand count ordered) Data: status of each contest selected for audit -- in particular, which contests if any has the Secretary of State selected for hand count? Data: the “risk limit” for each contest selected for audit. Data: the “error inflation factor” used to calculate whether the risk limit has been achieved Data: the “diluted margin” for each contest under audit, i.e., the vote difference between the lowest winning total and the highest losing total, divided by the total number of ballot cards in the county. This can be calculated from the CVR file. Data: for each contest under audit, the number of each type of discrepancy found in that contest by comparing Audit Board interpretations to the original CVR file. These can be calculated by comparing the entries in the CVR file to the corresponding entries in the record of Audit Board interpretations. The types of discrepancies are: Two-vote overstatements (Audit Board finds a vote for a loser where original CVR showed a vote for the winner) One-vote overstatements One-vote understatements Two-vote understatements Calculation: Using the formula for the stopping size given on Stark’s Audit Tools web page, see if the size of the sample checked so far exceeds the stopping sample size. With the following notations a risk limit g error inflation factor m diluted margin o2 number of two-vote overstatements o1 number of one-vote overstatements u1 number of two-vote understatements u1 number of two-vote understatements

the formula for the stopping sample size is:

stopping sample size = -2g(log(a) + o1log(1-1/(2g)) + o2log(1 - 1/g) + u1log(1+1/(2g)) + u2log(1+1/g)) / m)
sfsinger19103 commented 6 years ago

And here are the data specs pulled out of the protocol:

nealmcb commented 6 years ago

Note that the "source" draft document referenced above ("Public oversight protocol" on Google Drive) includes suggestions, updates and discussion, and is easier to read than the version here.

See also #601, which defines a similar set of requrements and includes some related queries.