Sequential page ordering via priority selection

[rogerhutchings]

@VVH raised this one with me again recently, and after talking it through, we concluded that:

• oversampling isn't really an issue on these kinds of projects outside of the initial media rush
• random selection ruins the sense of narrative

Currently SW subjects have a metadata.Priority field and a metadata.Page Sort field - @simoneduca, what's the difference?

Finally, @camallen / @marten, is priority sorted queueing available at the moment, and if so, is it a matter of copying the priority field to the corresponding set_member_subject and rebuilding the queue?

Otherwise, I imagine we'll have to traverse each set and add in prev / next links similar to what Sascha and Tom are doing on zooniverse/old-weather#95?

[VVH]

I'm looping in @eatyourgreens as we've been talking about this too and he seemed to have some thoughts.

[camallen]

I'm really starting to think we should just support priority and random via params in the api selector to allow the clients to choose which one they want (seems it not either or anymore):

• /api/subjects?subject_set_id=x&sort=queued for default (random)
• /api/subjects?subject_set_id=x&priority=true&sort=queued for priority ala here

That said, yes @rogerhutchings you'll still need the SMS priority fields copied from the subject as this is what the selection runs off.

[VVH]

Hey @camallen @rogerhutchings @eatyourgreens--I linked through to pull requests from the above, and don't see any issue #72 in closed issues. I wondered if it was this though: https://github.com/zooniverse/diagnosislondon/issues/52. In any case, can we find time to put this in production schedule and just get it done?

[camallen]

seems @saschaishikawa wrote something to do the backend work for us just recently https://www.npmjs.com/package/panoptes-subject-set-util

[VVH]

That looks good to me. Only question is will it be ok with skipping retired subjects? This will come up in AnnoTate and Shakespeare's World, because we've been serving randomly up till now. @saschaishikawa does your code allow for skipping of retired subjects? Or could it?

[camallen]

The client will still have to implement the page turning and UI change on retired data (i.e. client will request the retired subject status via the API). The API needs some work to support this data too (should be pretty quick) though.

[eatyourgreens]

zooniverse/Panoptes#1885 could affect this too (I came across it while updating the subject metadata for Diagnosis London to add prev/next subject links.)

[eatyourgreens]

@vvh Page turning is the 5th item on the todo list at zooniverse/diagnosislondon#72 It isn't closed because it hasn't been implemented there yet, either.

Diagnosis London does use prioritised subject queueing ie. serving the subjects in a predetermined order from first to last page. Is that what you were asking about, rather than browsing backward and forward through a volume by page? For that to work, you'd need many small subject sets, ordered by page number eg. the individual reports for Diagnosis London, or individual diaries in Operation War Diary. I think Shakespeare's World only has two large subject sets, each containing a collection of volumes, which would prevent you from using this approach.

[VVH]

Thanks @eatyourgreens you're right that Shakespeare's World has two large sets numbered 1-N in each. I was thinking we'd just go start to finish through the large sets, rather than smaller ones. We can do smaller sets in future, but this created problems with AnnoTate in the past.