zooniverse / scribeAPI

scribe API
MIT License
79 stars 25 forks source link

Fixing bug where retire_limit is never met in Mark #540

Open nonword opened 8 years ago

nonword commented 8 years ago

Originally workflow.retire_limit was an int (e.g. 3). Later, after implementing the completion_assessment_task we realized this should be a percentage (e.g. 0.75) to establish the target majority threshold when multiple conflicting completion_assessment votes have been cast. (In the future, we might modify this to take the time into account - so that a lot of early "yes" votes do not dominate later equally legitimate "no" votes.) In any case, we've configured our workflows with retire_limit 3, but we're checking that against the percentage of "no" votes when determining whether or not to retire. But [number of "no" votes] / [number of votes] will never be greater than 1 much less 3, so nothing is retired.

Also adding Emigrant discuss url.