Downloading results from rating round without creating new round

[slaporte]

Coordinators should be able to download rating results without creating a new round.

Currently, a rating round can only be finalized by advancing to another round. Some coordinators may choose to end with a rating round, or may want to see the outcome before configuring the next round. Perhaps there should be a config setting whether coordinators can view the voting results for in-progress rounds.

[mahmoud]

I think there's more discussion to be had here. For one, it's generally very rarely the case that one can generate an ordered result from a ratings round. We don't really support unordered winners, nor am I aware of this as a feature request.

For context, coordinators can currently see user progress, but to avoid tampering with results, they can't see specific ratings until the round is complete. For a variety of other intentional reasons, completing a round implies creating a new one.

Perhaps it would be useful to enumerate the actual user requirements here. Are coordinators just trying to check that the voting is going smoothly? Would showing a sample or distribution of votes work to that end?

[slaporte]

The case we have currently is from Wiki Loves Earth: they used Montage for a few rating rounds, then plan to do a final offline ranking round outside of Montage, so they needed to see a list of their finalists. (They don't need ordered results.) I can also imagine coordinators who want to see (or even share) the finalists from one round before they open up voting in the next in Montage.

We sort of support downloading results already -- they can create a new (dummy) round, and then download the results from the previous round. This is what WLE did, but it isn't a great user experience. The current flow requires coordinators to make a lot of decisions at once -- ending one round requires setting the name, jury, etc for the next. This makes sense in most cases, but poses a high barrier if someone just wants to see results after votes are all complete. I also want to minimize the number of times coordinators need to email us to figure out how to get their data if something unexpected comes up.

I have had a few coordinators ask to see results of their round, but I have not heard any coordinators concerned with tampering with results. There are some other ways to guard the integrity of the round, like making the audit log display prominently when a coordinator viewed a set of results. We could also have a "sealed" config setting for campaigns where this is a concern for coordinators.

[leilaz]

Thanks for creating this task and kicking off this discussion. A few thoughts on my end based on WLM 2016 and WLE 2017 experience:

• As slaporte mentioned, not all campaigns end in a ranking round. This can be because the specific country wants to have the final ranking round in an in-person meeting or through some other process that involves a lot of deliberation and back-and-forth between their jury members, or it can happen because simply they consider the rating round as the final round, for example.
• We can do education about best practices, and we do that as part of the work that the international team does in Wiki Loves Monuments, however, each country is responsible for their own jury process and giving them as much control as we can afford over how they run their jury process is important. Giving more control removes a lot of overhead from the international team and enables the countries to innovate in their jury process and remain self-sustained. These are all positive.
• To give you a sense of the frequency of requests we receive on this front: My conservative estimate is that 3 out of 5 coordinators ask for this feature. It's quite frequent.
• I like the idea of enabling this feature (download before closing the round or moving the "closing" of the round to the end of the current round, and not at initiation of the next round), but also offering a "sealed" config. I can see how that can be appealing to contest organizers, especially if you look at contests outside of the Wikimedia world. Based on my experience with Wiki Loves Monuments, I don't see that being picked up by the community of our users immediately, though. (I'm saying this to help you prioritize such feature).

[mahmoud]

I'm aware of the relative frequency, I'm just trying to understand the motivations behind the requests. I'd like to treat the use case of ending on a non-ranking round separately from ending on a ranking round outside Montage.

For instance, fleshing the external ranking round case out, I'd be curious how many organizers would be interested in uploading their final result to Montage for archival/reporting purposes. Technically that could be modeled now by starting a new ranking round with a single jury member, representing the jury as a whole. However, I'm sure we can do better than that.

As for unordered results, do we have any coordinators who've done this? Any results announced? We're working on a final report functionality and we may need to take this feature into account.

For record-keeping purposes maybe it's a good idea to have a drop down on campaign finalization asking the organizer what the reason is for early closing of the campaign and/or what, if anything, they're doing with the results. I think it's good to think holistically as Montage builds out more of a public face (#45) :)

[leilaz]

Re how many would be interested in archiving options: I don't have a good sense (but we can ask on the public list). My impression is that in a couple of years when/if Montage is completely part of their workflow, they will be interested but at this very moment, the final results are archived on Wikimedia Commons and that seems to be doing the job.

Re whether countries have actually finished with a non-ranking round: I don't know. We can ask at least the ones that had their final round in Montage as a non-ranking round.

And, I like the idea of asking the coordinator why they're closing a round if this doesn't get on their way too much or make the UI too busy. ;)

[papuass]

It seems that this has been implemented, can be closed.