[2.0-RC] summary.csv "Inactive Ballots Total" is different from the sum of each inactive ballot category

[yezr]

RCTab Version 1.4.0 config: 2013_minneapolis_mayor_config.json summary.csv: 2024-01-02_16-21-57_summary.csv

What Happens

I was documenting the list of changes made to the summary.csv in 1.4.0 and noticed a discrepancy. For each round we show 4 different categories of inactive ballots: Overvotes, Skipped Rankings, Exhausted Choices and Repeated Rankings. Then, we have an 'Inactive Ballots Total' sum. When I add up all of the individual categories I repeatedly get less than the 'Inactive Ballots Total' cell.

In this specific example the Round 1 inactives add up correctly sum(B59:62)=B63, but each of the transfer and Round columns after that do not sum correctly (column G & H for example).

What I would expect to happen

I would expect for each of the transfer and Round columns that the 4 inactive category totals sum to the 'Inactive Ballots Total' cell. I confirmed with @chughes297 that the 37 number in the transfer column after Round 1 is correct for this example.

[yezr]

One thing that might be helpful getting to the bottom of this is this google sheet from #782. That contest suffers from the same issue as this one.

It has a round by round explicit calculation of each ballot describing the candidates that it is supporting in each round. It also shows exactly when ballots go inactive and how much of their transfer value goes inactive when that happens. This is all in the Mat's Copy of Celeste Landry sheet.

You can see the issue described in this ticket in the summary file on the [1.4.0] summary.csv sheet in cell M31. It doesn't show .2222 of exhausted choices from ballots 26 and 11 after E is eliminated.

[artoonie]

Digging into the history here: This looks like it may have been, at some point, an intentional decision: https://github.com/BrightSpots/rcv/issues/633

The number of "undervotes" is not shown. The breakdown only shows people who either made a mistake (overvotes, skipped rankings, repeated rankings) or were forced to undervote (exhausted choices), not people who elected to not use all their choices.

This is a simple change to make if we decide we want to add "Inactive Ballots by Undervote" (or similar language).

[yezr]

Finally looked at the previous discussion in the original expanded summary.csv stats here. Talking with @chughes297 here is the logic we'd like to use.

Whether a ballot has used all of their rankings or not, if all of their ranked candidates have been eliminated they should be included in the Inactive Ballots by Exhausted Choices bucket. With these exhausted ballots now included, then the different Inactive by XXXX (rows 59:62) should sum to Inactive Ballots Total (row 63)

I had mistakenly never assigned this to the RCTab 2.0 project. I added it and made it a P0!

[yezr]

from that comment, this is related

Overvote Decision is Skip To Next Rank: 1: Apple 2: Banana 2: Cherry

When apple is eliminated, does this ballot become inactive-by-overvote or inactive-by-exhausted-choices? I'm inclined to say inactive-by-overvote but I can see an argument for either.

It would be inactive-by-exhausted-choices since the overvote is "allowed" because of permissive Skip To Next Rank. In this case there happens to be no more rankings and therefore exhausted.

[artoonie]

Whether a ballot has used all of their rankings or not, if all of their ranked candidates have been eliminated they should be included in the Inactive Ballots by Exhausted Choices bucket. With these exhausted ballots now included, then the different Inactive by XXXX (rows 59:62) should sum to Inactive Ballots Total (row 63)

I see benefit to understanding when a voter was unable to rank more candidates versus when they chose to not rank more candidates. Is this distinction important to you? If so, I can add another category instead of combining them.

It would be inactive-by-exhausted-choices since the overvote is "allowed" because of permissive Skip To Next Rank. In this case there happens to be no more rankings and therefore exhausted.

I believe this is indeed the case now, based on that discussion. Let me know if I should verify this.

I had mistakenly never assigned this to the RCTab 2.0 project. I added it and made it a P0!

Will get to it!