Allow debate ballots to be submitted individually, rather than all at once

[czlee]

The current public ballots form requires one adjudicator to submit all ballots on behalf of a panel. It would be nice to provide an option whereby each adjudicator only submits their own ballot.

This might require database schema changes.

[tienne-B]

This is one of the first things I wanted to change (along with motions). At first, I didn't think it would require a schema change, but it would after-all. It would be a preference to whether to have one person submit them all or each individually. As a side-effect, debate identifiers should become debate-adjudicator identifiers so that each ballot could be tracked.

In this model, all ballots for a debate would need to be confirmed individually. A new icon would be useful in the admin interface would be useful to distinguish between debates without any ballot, with a ballot, and with all ballots (with variations for statuses; such as for when an incomplete set has one confirmed and another non-confirmed)

SpeakerScore and TeamScore would not be able to reference a specific BallotSubmission (would need a many-to-many relation, but that shouldn't be necessary) However, BallotSubmission can be used ballot-by-ballot (I'd add a DebateAdjudicator foreign key to make the submitter explicit) Instead; those objects would be created when all the ballots are entered. The difficulty would be in resolving potential inconsistencies between ballots, such as the motion selected or the order of speakers. When a director confirms the set to create the non-adjudicator score records, it should check for these problems. All voting adjudicators would need to input all information to reveal inconsistencies (if any). The verified information would be used for Speaker-/TeamScore and for the motion field (I'd suggest having it as a field under Debate)

The big change would be in the interface though, as result-entry pages would have to support many BallotSubmissions and have just one form from the formset for the adjudicator entry.

[czlee]

Yeah, there's more than one way to do this, and it's definitely nontrivial, as you've noticed. The hard part isn't so much figuring out how this system should work, as it is how it should fit in with the existing ones. As usual, I don't want existing workflows to change. So consensus ballots should continue not to use SpeakerScoreByAdj, for example, nor should they be associated with any adjudicator directly: if you change the chair of a consensus ballot debate that already has a result, that shouldn't attempt to cascade-delete any data about the result of the debate. (With voting ballots, this is unavoidable.)

Ordinarily, our "no change to existing workflow" rule would require us to maintain the current workflow, where tab teams that are using only paper ballots can just enter all three ballots on the same page, as they can now. But unless @philipbelesky objects, I don't mind making an exception here. The current workflow, where all ballots for a debate are entered simultaneously, is something we inherited from the code base in 2012, and I think there's a reasonable case that even for tab teams it might be better to think about ballots as individual ballots rather than as grouped with a debate.

One thing that shouldn't be too difficult, but I would want to think through rigorously: How does Tabbycat know when to populate the debate result? Like, how does it track this status, and what does the database look like while some but not all ballots are in, and how do the views deal with this? Does this break any assumptions that other code makes about database consistency? Should tab staff have to confirm the set as a separate step, or should it be automatic upon confirmation of the final ballot?

[philipbelesky]

No principled objection to the change. The one concern would be splitting ballots across multiple pages does seem to be a less efficient and potentially more error prone entry process. One way around that might be to only split ballots this way when the preference is enabled, or perhaps allow the single page to have blank ballots data along with indicators on the results page as to how many of the set have been entered.

[tienne-B]

There would not be a preference to change this new behaviour. At its base, this change would make each individual ballot have its BallotSubmission object (and so each ballot would be confirmed/discarded separately). The admin/assistant ballot entry views would be a formset for all BallotSubmissions relating to a debate, rather than the set of scoresheets related to a single ballot submission (the term "ballot" would be clearer).

In that, the major workflow change would be that all voting adjudicators would need to submit their ballot individually; the chair doesn't enter them all. There wouldn't be any major changes in the admin/assistant workflow (as all ballots to a debate is still on one page). The big difference I see is that there would be check-boxes for ballot status on each ballot (but could be hidden behind the "Submit" button for assistants) and that speaker positions would have to be specified for each ballot). And yes, the entry pages would show how many ballots are inputted / outstanding for each debate.

To clarify, consensus ballots would be unaffected by this as it is already one-ballot-per-ballot-submission. Also, the aggregate objects (Speaker-/TeamScores) would be created once the debate is confirmed (they will have to have the ballot_submission field removed). Tabbycat would know when it is complete when all the voting adjudicators' ballots are confirmed. So each ballot would create SpeakerScoreByAdj objects (which may be inconsistent with each-other). If the aggregate objects are not created; non-entry views would ignore the debate's results.

[philipbelesky]

Why would there not be a preference to retain the behaviour of chairs entering in all the ballots individually if using online submissions? I'm open to the option but that is a big departure from the existing workflow that people are acclimatised to and it also seems more risk prone - ie it relies on all voting panelists (rather than just one) having internet access, knowing their URLs, understanding the process, etc. Or did I misunderstand that part of the comment?

[tienne-B]

I misunderstood you. Thought you were suggesting a preference for the admin interface. But yes, there would be a preference to allow adjudicators to submit multiple ballots at once.

[tienne-B]

Note: Seems to have been discussed previously as part of #30.

[czlee]

I don't even remember discussing that, lol. This is why we run these discussions on a public issue tracker, I guess…

[0zlw]

Hi all, I seem to be in the habit of surfacing some old issues this afternoon. I think this issue needs to be revisited in the context of online debating where some of the historical assumptions around the way debate workflow happens needs to be revisited based on my experiences with running a pre-Australs tournament this weekend.

We were running a Zoom/Discord tournament, with specific Discord text chats for judges to submit their marksheets for each room.

Our intended workflow for judges was the following:

1. We created a google doc marksheet for judges to fill in (https://docs.google.com/document/d/1dastD46k-oRcrukMPPGC97QFvBN3-JlVh_07auLok9U/edit). Judges were meant to make a copy of the marksheet or download it.

2. At the end of the debate, judges were meant to individually arrive at their decision, fill in their google doc marksheet then take a screenshot and upload this to the relevant discord channel.

3. Once the marksheets were completed the panel should view each other's marksheets, vote and deliberate as usual.

4. During the OA one of the judges should submit the collective marksheets of all the judges.

We treated either one of the online marksheet or the google doc as a source of truth (i.e. we were not relying on double confirmation of every marksheet before confirming the draw for the next round).

In practice we found that:

1. Compliance with uploading the marksheets to google docs was not 100% - this hovered at around 80% over the course of the tournament. We made general reminders, although we didn't seek to individually tell judges who were doing this do it.

2. The issue in 1 notably affected how marksheet entry for panels was working. In many cases we would have 2 out of 3 document marksheets come in from a panel, but often be missing 1 of them, or missing 1 of them for an extended period of time.

3. In some cases, it appeared that panelists were reading off their scores to the chair to enter into the online marksheet, introducing time delays and another step where errors could be introduced.

4. It seems that because panelists were not always compelled by the chair to submit a paper marksheet before entering deliberation, often the paper marksheet was only completed after deliberation had started.

We were running a pre-Australs tournament from New Zealand that attracted a number of teams from Asia who I think were often not experienced with Australs norms (i.e. voting not consensus) and could be fixed with continued education and upskilling by tab team but I think some of the issues we were having speak to the fact that it could be worthwhile to think about what the best workflow is for online debating ballot entry and confirmation.

What immediately comes to mind is potentially facilitating a way for judges to individually submit marksheets individually in a way that then lets the other panelists see it, and then eventually requiring one judge to "confirm" the marksheets on behalf of the panel but I am not sure what other changes I would want to see.

This would also be helpful from a tab team perspective. Often for online debating, the marksheets from different panel members "arrives" separately when they upload it, rather than arriving together in a bundle. Being able to enter partial marksheets (i,e, save 1 at a time on a panel) would be improvement.

I'm sure this is a fairly complex area of Tabbycat workflow and would be quite a significant piece of work to implement so would be interested to hear your thoughts (and thoughts from anyone else running online debating).