Revise revisions UI

[mbernst]

The user comment we've kept getting is a confusion about where their results have gone to. We've been discussing this on-and-off for a couple months at least, so I'd like to propose a concrete solution.

THE PROBLEM

Requesters edit their tasks after launching. We have a decent interface for making this possible. It's much better than current alternatives like having to take down the task and restart it, because of some smarts that @dmorina built in to automatically port existing tasks over to the new revision.

However, it makes some things more complicated. Specifically, each time the user edits the task design, it creates a new revision in our back-end. This might, for example, change the names of questions, add new ones, or delete some. Currently, what our system does is make a clean break where the user can flip between different revisions in the results view.

However, a lot of users don't know to go look at the past revisions, even with the addition of a call to action that helps them. This proposal is focused on revising the system model to bring it more into line with the user model.

THE PROPOSAL

This problem is not unique to Daemo: other platforms such as Qualtrics and Google Forms have to deal with the schema changing as data arrives. My proposal is that we adopt an approach inspired by Google Forms, which merges all the revisions into a single review interface.

Specifically:

• We have one unified results view on the web by default, rather than a series of different views per revision.
• To populate that unified view, we take the union of all columns in all revisions that had at least one taskworker submission, much like Google Forms. Again inspired by Google Forms, if a taskworker submission doesn't have a particular column, it's left blank.
• If a column was left unchanged between revisions, it isn't copied (thus the "union of all columns")
• We sort the columns left to right essentially by revision. So the leftmost columns were the earliest submissions, and the rightmost columns were the latest submissions.
• We would add a "Revision" column that indicates which revision of the task the submission came from.

For expert users, we can keep the dropdown per revision as we have it now. This will allow a user to drill down into a specific revision if they want.

PROS

I think this will be much clearer to requesters, since their mental model doesn't involve "revisions": they just edited the task, they didn't "make a revision". It's also a familiar model from other interfaces.

CONS

It will potentially make the results view much wider, with more columns. I think I'm willing to swallow this tradeoff.

CSV compatibility

Ideally I would advocate doing the same merging in the CSV download as well. Perhaps a merged CSV in addition to the individual revision CSVs...but I would advocate just the merged one.

[markwhiting]

I think we've talked about this as a dominant option in the past and I'm in favor.

Regarding the con of being visually harder to consume, one option would be to allow minimizing columns, basically something like if a heading is clicked, the columns width toggles to 1.5em or so (enough to see there's a result), and then any contents are made visible in a tooltip if hovered over.

[dmorina]

the other option would be to teach them what a revision is by explaining that when they edit their design. and anything can change price, title, number of assignments or any other field so a revision based grouping totally makes sense because you will end up with so many repeated values in this huge table otherwise. Also implementation wise this approach of merging these non static fields is "not clean" and can be slow.

so my proposal would be: 1) let the user know when they are about to create a revision 2) hide "empty revisions" i.e the ones that have no responses 3) potentially convert the drop down to tabs so that they are aware of these revisions

[markwhiting]

I think these changes are also a good improvement over what we have.

I would clarify that I think @mbernst was suggesting that fields would not repeat. Instead they would overlap where possible, and and remain where unique.

[mbernst]

Here's my motivation for this approach as opposed to "teach them": users not only have a mental model that doesn't include revisions, I can't make a case for why their mental model needs to include revisions.

It seems that in the vast majority of cases, requesters will get by just fine without needing to think about revisions as they edit tasks and review results. We should definitely make it accessible to the experts who need it, but I would claim it's very rare.

[markwhiting]

I'd argue on the side of not making a separate experience for experts. For example, if a user knows of the shortcomings of our approach (whatever we choose), they should be able to take a standard action to counteract those shortcomings in the way that is relevant to their situation (e.g. decide to edit or duplicate the task depending on what they need).

[dmorina]

im not saying let's just "teach them", it's more like let's iterate improve this as much as we can and if there's something (a tiny bit) left to teach then that's ok. I feel like most of the complains were related to system generated revisions i.e prototype task which I totally understand how it can confuse people, or the empty revisions. There are many cases where users discard data when they make a mistake in early revisions and also like to do these in "runs" so such grouping is fairly logical.

To minimize errors implementation has to be as close as possible to the way they user uses our platform what you call "user mental model" which means either remove revisions entirely if you are certain that a user doesn't seem to need them which is a yuge backend change (and not if favor of doing) OR we can leave backend as is and treat front end as we would treat any client and let it transform the data as it wishes to, in that case you will end up with something extremely slow and not feasible

I also agree about not having different experiences for different users (at least not in the same view), try to make it better for the normal user and everyone else will be happy as well

[shirishgoyal]

If requesters do not have mental model for revisions, can we instead keep it simple and only allow them to fork project and launch new one than handle this complexity of revisions which is only serving very few use-cases (forcing single user for each task, not launch completed task again, pricing changes for versions etc.)

[mbernst]

I'm fine with trying to get rid of the "empty" revisions --- e.g., p. tasks, changing price --- and seeing if that helps. I suspect that it might not.

I disagree that a user mental model w/o revisions requires a system model w/o revisions. Of course Google Forms maintains revisions behind the scenes — it's just that users don't really need them.

If we have evidence that people really treat these batches separately, that would be a good argument for keeping it. But creating a new revision for every tiny tweak seems like it's causing more harm than good?

[mbernst]

Moved to /collective repo issues