Add participant age restriction

[Speldosa]

I know that you can restrict the age of your workers when creating a HIT via the Amazon Mechanical Turk web page. However, is this possible to do when creating a HIT using psiTurk? Or would it be possible to modify/add a new criteria to the HIT afterwards somehow?

[deargle]

You cannot do it at present with psiturk, but you can modify psiturk to do it. Look at recent-ish commits on master for Master Worker qualification and min num approved requirements for where and how to modify the code.

On Sun, Apr 1, 2018, 4:01 PM Martin Asperholm notifications@github.com wrote:

I know that you can restrict the age of your workers when creating a HIT via the Amazon Mechanical Turk web page. However, is this possible to do when creating a HIT using psiTurk? Or would it be possible to modify/add a new criteria to the HIT afterwards somehow?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NYUCCL/psiTurk/issues/312, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHsfaeWU9FCe2TGHBupztX3q4uoIxXpks5tkU4bgaJpZM4TC_Mo .

[Speldosa]

So this would be the only way of doing it then?

Sure, I could take a look.

[deargle]

Only way as far as I know.

On Sun, Apr 1, 2018, 4:20 PM Martin Asperholm notifications@github.com wrote:

So this would be the only way of doing it then?

Sure, I could take a look.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/NYUCCL/psiTurk/issues/312#issuecomment-377821236, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHsfZsTmqPzu6yYTL38vsgWeqIn2hXxks5tkVKugaJpZM4TC_Mo .

[Speldosa]

I've been digging around a bit in the boto API (version 2, which psiTurk is using, and version 3). There seems to be classes that deals with certain qualifications directly, such as number of HITs completed that @deargle referenced above. It's still not clear whether the Premium Qualifications can be accessed via the API though.

I've only seen one discussion directly addressing this issue, and there it's suggested that you should create your hit via the Amazon webpage (with your custom qualifications) and then tell boto you want to use that specific HIT.

[Speldosa]

I've noticed that there are services like TurkPrime and Prolific where you can specify what type of participants you want and then send them to an external URL where they'll eventually will get a URL/code that they can use for payment.

This would be suitable for my needs as well. You could simply rewrite default.html (found in the example) to invoke the lab mode (discussed in issue #95; also see this Google Groups thread) and thanks-mturksubmit.html to include the payment code.

I realize that this is somewhat of a hacky solution where several things probably could backfire (for example, I realize that it's hard to keep somewhat tech-savvy participants from re-starting your experiment using this method), but it seems like my best option for getting this thing up and running.

[deargle]

I have thought lightly about using a site like Prolific with psiturk. Can TurkPrime and Prolific generate random strings for your urls? Psiturk requires a unique workerid : assignmentid combo for each participant, which are passed through the url.

If it can't to dynamic url generation, then yes, you can do the same in default.html, where they type in their workerid or however they are identified on the platform, then a big button to press that creates a custom url for them with random entries for the assignmentid. Or if Prolific can insert the workerid only into the url, then you can call a custom psiturk route, which receives the workerid, and then auto directs them to the next page with randomly generated assignmentid, without the intermediary requirement of them visiting default.html and pressing a button.

And if you're using lab mode, you wouldn't hack thanks-mturksubmit.html. You would hack complete.html.

[deargle]

And age is really a "premium qualification"? That's dumb.

[Speldosa]

From what I can tell, neither TurkPrime nor Prolific can add any random strings to your URL (TurkPrime does it if you run experiments exclusively on Mechanical Turk, but that's effectively like running the experiment on Mechanical Turk yourself, and you don't get any extra filtering options here).

I guess Amazon wants to get some extra profit by having things like age and gender as premium qualifications, which I guess is up to them, but it's strange that you can't really access them via the API (or so it seems).

[jodeleeuw]

Prolific will add the participant and session ID to your URL if you want.

On Wed, Apr 4, 2018, 7:27 AM Martin Asperholm notifications@github.com wrote:

From what I can tell, neither TurkPrime nor Prolific can add any random strings to your URL (TurkPrime does it if you run experiments exclusively on Mechanical Turk, but that's effectively like running the experiment on Mechanical Turk yourself, and you don't get any extra filtering options here).

I guess Amazon wants to get some extra profit by having things like age and gender as premium qualifications, which I guess is up to them, but it's strange that you can't really access them via the API (or so it seems).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NYUCCL/psiTurk/issues/312#issuecomment-378567765, or mute the thread https://github.com/notifications/unsubscribe-auth/AAkWRJffvqqGXeJ95x6yOIjP3SQyn4dYks5tlK4XgaJpZM4TC_Mo .

[Speldosa]

@jodeleeuw Nice! I didn't see an option regarding that when creating a test study in Prolific. Is it obfuscated somewhere?

---

Also, I just realized that you can create a HIT directly on the Amazon Mechanical Turk website (I don't know why I didn't think about this before) and use the "Survey link" option to send the participants to your psiTurk server. With this method, you can also access and use the Premium Qualifications. It also seems like a string with information about the participant is added to the URL.

[jodeleeuw]

Yes, it is kind of buried in the new interface. It used to be obvious. Here's the relevant doc: http://help.prolific.ac/getting-started-launching-your-first-study/how-to-get-the-participant-id-from-a-query-string

[nicoa47]

@Speldosa Regarding your suggestion to use AMTs "Survey Link" option directly, how do you obtain the relevant psiturk link to the experiment? AFAIK you'd need to use "hit create" in the psiturk console, but wouldn't this inseparably create a link on AMT - and thus make the ad visable for ANY age range?

[deargle]

on mobile, but no, you would not use "hit create." See earlier links to discussion about "lab mode." Gist is, you enter a direct link to your experiment server for participants, on the aws mturk web interface. Your psiturk server is always accessible via the direct link, whether you create a hit from console, web, or nowhere.

[deargle]

alternative is to do something like what requesters had to do before premium qualifications existed -- give a presurvey where you survey things like age, and then create your own custom qualifications. I just added some custom block / require functionality to master, extending off of what @fredcallaway was talking about and doing in a more recent issue.

[fredcallaway]

Awesome, great work @deargle!

[nicoa47]

Edit: I put the following question in the PsiTurk Google group to avoid Off Topic: https://groups.google.com/forum/#!topic/psiturk/ihB7bCHJL00

@deargle Thank you for the fast response! I was not aware that the other modes work as direct links.

I'm still confused how the exact link should look like. I assume it's of the shape {IP}:{port}/{html-site-name}?hitId={xxx}&assignmentId={yyy}&workerId={zzz}&mode=lab ({...} are variables)?

Are all three ID's in the URL obligatory? If so, how to ensure participants have unique ID's in the URL via AMT? Or, if the link stays the same, how ensure that they don't pass the link around?

[deargle]

Or, if the link stays the same, how ensure that they don't pass the link around?

psiturk will block a worker from taking the task more than once -- that's a core feature

Are all three ID's in the URL obligatory? If so, how to ensure participants have unique ID's in the URL via AMT?

In short, AMT generates the url based on the ad url base that we set when posting the experiment, feeding in the current hitId, assignmentId, and workerId when someone "accepts" the hit.