NYUCCL / psiTurk

An open platform for science on Amazon Mechanical Turk.
https://psiturk.org
MIT License
277 stars 140 forks source link

error running "worker approve" - maybe related to code version discrepancy #385

Closed yang-yangfeng closed 4 years ago

yang-yangfeng commented 4 years ago

I'm trying to run (with psiturk version 2.3.3): psiturk -e "worker approve --all"

This has worked for me in the past, with errors only occurring for cases in which there was an actual issue with mTurk recognizing the assignment as complete. Now, none of my outstanding assignments are being approved/status is not changing to 5. This is accompanied by an error message that looks like

Response(Operation: approve_local_assignment | Status: error | Exception: An error occurred (RequestError) when calling the ApproveAssignment operation: Assignment XXX does not exist. (1576537328497 s))

However, if I just use the aws mturk CLI to directly approve, it goes through fine.

One thing that might be related is I changed code versions before approving all of the older assignments - so there are unapproved assignments of version 4.0 and version 4.1 at the same time. For one of the HITs, there is a mix of versions (I believe I forgot to update the config.txt until after I had launched the HIT - otherwise, there's also something weird about this?).

I think I can just go ahead and approve remaining assignments with the CLI, so it's not a huge deal - but it looks like status codes won't update unless approval happens through psiturk - and just wanted to make a note in any case.

deargle commented 4 years ago

Which aws mturk cli are you referring to? That error messaage text is coming from boto3. Does the XXX assignment id shown in the error message match the one you are passing? And are you in the correct "mode" in the psiturk cli?

On Mon, Dec 16, 2019, 4:14 PM Yang-Yang Feng notifications@github.com wrote:

I'm trying to run (with psiturk version 2.3.3): psiturk -e "worker approve --all"

This has worked for me in the past, with errors only occurring for cases in which there was an actual issue with mTurk recognizing the assignment as complete. Now, none of my outstanding assignments are being approved/status is not changing to 5. This is accompanied by an error message that looks like

Response(Operation: approve_local_assignment | Status: error | Exception: An error occurred (RequestError) when calling the ApproveAssignment operation: Assignment XXX does not exist. (1576537328497 s))

However, if I just use the aws mturk CLI to directly approve, it goes through fine.

One thing that might be related is I changed code versions before approving all of the older assignments - so there are unapproved assignments of version 4.0 and version 4.1 at the same time. For one of the HITs, there is a mix of versions (I believe I forgot to update the config.txt until after I had launched the HIT - otherwise, there's also something weird about this?).

I think I can just go ahead and approve remaining assignments with the CLI, so it's not a huge deal - but it looks like status codes won't update unless approval happens through psiturk - and just wanted to make a note in any case.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NYUCCL/psiTurk/issues/385?email_source=notifications&email_token=AAI6Y7MJZZ3MJGFPANK2ZXTQZADUJA5CNFSM4J3TI6QKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IA4NSEQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI6Y7KL5Q2MLW7HGLRFCSDQZADUJANCNFSM4J3TI6QA .

deargle commented 4 years ago

Feel free to reopen with a reproducible case