Feature: Whitelist and blacklist qualifications

[fredcallaway]

I need this myself, so I can make a pull request for it. Just wanted to ask for input before I implement it. Also, if someone could point me to where the hit_config argument to MTurkHit.configure_hit is initially created, that would be most helpful.

User interface:

# config.txt
[HIT Configuration]
require_quals = QUALIFICATIONIDSTRING,ANOTHERQUALIFICATION
block_quals = YETANOTHERQUALIFICATION
...

Implementation in MTurkHit.configure_hit:

for qual_id in hit_config['require_quals'].split(','):
    quals.append(dict(
        QualificationTypeId=qual_id,
        Comparator='Exists'
    ))
for qual_id in hit_config['block_quals'].split(','):
    quals.append(dict(
        QualificationTypeId=qual_id,
        Comparator='DoesNotExist'
    ))

[deargle]

You could read it straight from config, https://github.com/NYUCCL/psiTurk/blob/993f114c060f9cb43e84da9a64a695bc03023025/psiturk/amt_services.py#L300 instead of from hit_config. as for finding where that's defined, a general approach is to use something like git grep configure_hit, or search for configure_hit on github for this repo, which would point you to see that it's called by create_hit, https://github.com/NYUCCL/psiTurk/blob/993f114c060f9cb43e84da9a64a695bc03023025/psiturk/amt_services.py#L358-L362 which in turn is called here-ish https://github.com/NYUCCL/psiTurk/blob/07aceda553658b03bb6c6880368ec5f59fe2e0b0/psiturk/amt_services_wrapper.py#L799-L810, which calls _generate_hit_config to make the hit_config, https://github.com/NYUCCL/psiTurk/blob/07aceda553658b03bb6c6880368ec5f59fe2e0b0/psiturk/amt_services_wrapper.py#L897-L912

Or use import pdb; pdb.set_trace() and get a stack trace which would point you in the right direction.

With this implementation, please also write a few unit tests, testing things like making sure that the expected require_quals and block_quals are set when create_hit_type is called here https://github.com/NYUCCL/psiTurk/blob/07aceda553658b03bb6c6880368ec5f59fe2e0b0/psiturk/amt_services.py#L290-L297, via amt_services_wrapper, please and thank you

[fredcallaway]

For what it's worth, I implemented it on my fork and it appears to be working (using the test by deployment method). Unfortunately, I don't know much about testing practices and I don't imagine I'll find time to try to understand the current testing suite well enough to add a unit test for this functionality. But if anyone else has this request, hopefully they'll be able to find this issue.

https://github.com/fredcallaway/psiTurk/commit/55f7460cdab675dc04203a68e9d9c25c2ab9f80f

[deargle]

Note to self before I close this that qual ids differ between sandbox and live, so maybe have separate entries in config.txt for sandbox vs live quals. seems a pain though.

[deargle]

Closing because this is generally implemented, but would still be cool to have a config-easy way to differentiate between sandbox- and live-mode qualifications.