votingworks / vxsuite

https://voting.works
30 stars 6 forks source link

Perform scale testing and settle on system limits #5116

Closed arsalansufi closed 6 days ago

arsalansufi commented 2 months ago

As a first step, product will identify a list of dimensions for which we need to define limits (e.g., number of precincts, number of contests, contest title character length, number of candidates in a contest, and so on), but not necessarily all the limits, themselves.

The limits that we pick for cert do not need to be future-proof for all possible customers as we'll almost certainly be submitting a mod for whichever state we next enter.

Software engineering and product will ultimately have to work together to define these limits.

arsalansufi commented 2 months ago

A relevant perf improvement idea from @eventualbuddha: https://votingworks.slack.com/archives/CEL6D3GAD/p1721847925442059

mattroe commented 2 months ago

List of limits we will set and some initial suggested values, competitive numbers, and orders of magnitude I think we should test to: https://docs.google.com/spreadsheets/d/11nClik0hRYJYb0riY6nvtOKJysASdTFBrkAuss6QSio/edit?gid=0#gid=0

The exact values here are pretty flexible until we start modifying the certified system for a specific deal. I suggest we automate creation of election definitions at the listed small, medium, and large orders of magnitude for these values. We can see what breaks in these tests and iterate on these limits from there.

mattroe commented 2 months ago

These are the general election definition system-wide values that we plan to set limits for based on thinking here: https://votingworks.slack.com/archives/CE7LM9LQ2/p1722993765063239

mattroe commented 2 months ago

This doesn't include some known HW limitations like ballot size, batch size on a scanner, etc. that we will also specify.