search

YaleDHLab / ensemble-at-yale

Crowdsourcing the transcription of Yale playbills - http://bit.ly/ensemble-at-yale
http://ensemble.yale.edu
MIT License
6 stars 4 forks source link

Transcribers get bumped to a different program instead of working through the one they start #157

Closed lindsaymking closed 5 years ago

lindsaymking commented 5 years ago

If a user wants to work on one program, Ensemble lets her mark all the fields and then begin transcribing that same program. However, Alex and I have discovered that at a certain point (for me it was at 9 or 10 fields transcribed) Ensemble bumps you out to an entirely different program to transcribe. It has happened multiple times, but perhaps is a high enough number that we didn't encounter it in user testing. Is this a setting somewhere? We do want users to be able to get the satisfaction of following through on one whole program.

duhaime commented 5 years ago

Hmm, that's interesting. Is this for records that no previous users have marked?

The one case I can think of where this might occur is the following scenario: some users mark fields in a record, and some other users transcribe those marks. Then a new user comes and marks the same, but they don't add any new mark types to the set of marks for the record. The user then attempts to transcribe, and their nth transcription retires the last unretired mark from the playbill. This then marks the playbill as fully marked and transcribed.

If your records weren't retired from marking and transcription, though, there must be some other logic wormhole that leads to this situation. Do you have a sample record on which this occurred? If so, we can use the subject_set_id of that record to study the situation from the db...

lindsaymking commented 5 years ago

If you go to the Department of Drama era and look at the program for Coronation, it appeared that I was the only person to work on that program, so it might be a good example. It definitely happened twice--when I initially finished marking and then chose to transcribe the same program, and then when I went back to try to transcribe more. Alex said that it happened to her with Beware of the Bull, at the bottom of that same era.

pleonard212 commented 5 years ago

We've just tested this on "Hotel Universe", 1947. This is a simple one-pager, that had no marks on it before we started. We marked it up and transcribed it and didn't hit the bug... which is not to say it doesn't exist! One theory is that some programs have some marks that have (invisibly) been "retired" and don't need transcription. There's no way for the user to know this. And so when she is only partly done with the apparent transcription task, the system kicks her to a new play because it regards all the marks as now done.

It seems the right way to fix this (if we are correct in our theory) is a possible UX enhancement:

In the Mark workflow, disable/grey out the button for the kind of kind of mark we already know (because it's already both marked and transcribed). This would not work for the Actors and Staff members, however, because they always could be more of them...

Why don't we bring @alexokeefe by the lab some time and explore both whether this is really happening the way we think, as well as brainstorm some UX solutions?

duhaime commented 5 years ago

Link for Hotel Universe: http://ensemble.yale.edu/#/mark?subject_set_id=5919e055adb8170833fb5122

lindsaymking commented 5 years ago

I just did some transcribing in Hotel Universe, starting at the top of the program, and after I typed in Norman Rose and the actor's name, the system bumped me to another program.

alexokeefe commented 5 years ago

I'm happy to come by to look at it together! I'm not sure how everyone's week looks, but would it be possible to set up a time somewhere between 1:30-3:30 on Thursday? If we can meet before the first transcribe-a-thon that would be ideal, but I understand if not.

duhaime commented 5 years ago

Thursday 1:30-2:00 or 3-4 works for me!

duhaime commented 5 years ago

Ah, so when users click to transcribe a playbill they've just marked, the app requests the first n unretired marked fields from that playbill, where n is specified in the transcribe config.

That value gets queried by the fetch subjects mixin, which queries the workflow model.

I think the easiest fix here is just to just update the value in mongo itself: db.workflows.update({'name': 'transcribe'}, {$set: {'subject_fetch_limit': 100}})

I'll also change the value in the config file for posterity...

pleonard212 commented 5 years ago

That's really interesting... and so it'll iterate thru the (less than 100) subjects it returns, and bounce you to the next program when it reaches the end of that queue?

duhaime commented 5 years ago

Exactly. To prevent having to query for the next subject to be transcribed after each submission of a new transcription, the app batches its requests, so requests 10 (or now 100) all in one fell swoop.