webis-de / mturk-manager

An alternative front end for Amazon Mechanical Turk
GNU General Public License v3.0
12 stars 4 forks source link

Some workers are limit-blocked without having worked on a task #62

Open johanneskiesel opened 3 years ago

johanneskiesel commented 3 years ago

I now got two mails from workers that said they got this on their first attempt to work on a task:

Temporary limit reached. Please return this HIT.

We decided for this procedure, so we don't have to reject many HITs and affect your ratings.

Thank you.

I remember there was something that even for them to attempt a task (and then return it) would count (which, I think, is correct). But this seems different? I can provide the Worker ID for debugging on a private channel.

johanneskiesel commented 3 years ago

I just got confirmation from the worker that they accepted one (1) HIT and then returned it, before the message came for the next HIT. Our limit is, however, five (5).

johanneskiesel commented 3 years ago

In any case... I guess it would be helpful to have the workers show up in the list of workers for the task even if they have not completed a single HIT yet, just so that I can reset their limit (I would want to do this in the case they accepted and returned 5 HITs).