search

NYUCCL / psiTurk

An open platform for science on Amazon Mechanical Turk.
https://psiturk.org
MIT License
277 stars 140 forks source link

Experiment exchange examples not working #361

Closed yang-yangfeng closed 5 years ago

yang-yangfeng commented 5 years ago

Mainly making this issue because I'm not sure what would be the best way to put this info in the readthedocs

I've noticed that out of the 5 or so examples I've tried running, only the official psiturk-example works as expected. First, the server on command fails because these repos don't seem to include the custom.py file in the top level directory (or anywhere?) - copying the custom.py file from psiturk-example fixes this. Second, it seems like there must have been some recent changes in how the html must be formatted (e.g. certain attribute requirements) - I simply edited certain lines from the ad.html and default.html template files to match those from psiturk-example.

jacob-lee commented 5 years ago

Probably requires closepopup.html and thanks-mturksubmit.html too.

These sort of changes have been a headache, when re-running old experiments, or code based on them. The experiment exchange is only going to be useful if there is some kind of versioning system in place.

On Tue, Jul 23, 2019 at 1:48 PM Yang-Yang Feng notifications@github.com wrote:

Mainly making this issue because I'm not sure what would be the best way to put this info in the readthedocs

I've noticed that out of the 5 or so examples I've tried running, only the official psiturk-example works as expected. First, the server on command fails because these repos don't seem to include the custom.py file in the top level directory (or anywhere?) - copying the custom.py file from psiturk-example fixes this. Second, it seems like there must have been some recent changes in how the html must be formatted (e.g. certain attribute requirements) - I simply edited certain lines from the ad.html and default.html template files to match those from psiturk-example.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NYUCCL/psiTurk/issues/361?email_source=notifications&email_token=AAM5SQIB5QWNLDVHAZZIUC3QA477TA5CNFSM4IGHGCF2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HA7VTPA, or mute the thread https://github.com/notifications/unsubscribe-auth/AAM5SQN6L23GYXAQP64MN4DQA477TANCNFSM4IGHGCFQ .

deargle commented 5 years ago

I've been mulling having some kind of "psiturk experiment upgrade" command which would copy in the commonly-missing files, and update the database. I saw sqlalchemy alembic the other day which can do db migrations/upgrades via python. Also a try-catch which would provide an error message suggesting to run the upgrade command.

On Tue, Jul 23, 2019 at 1:43 PM jacob-lee notifications@github.com wrote:

Probably requires closepopup.html and thanks-mturksubmit.html too.

These sort of changes have been a headache, when re-running old experiments, or code based on them. The experiment exchange is only going to be useful if there is some kind of versioning system in place.

On Tue, Jul 23, 2019 at 1:48 PM Yang-Yang Feng notifications@github.com wrote:

Mainly making this issue because I'm not sure what would be the best way to put this info in the readthedocs

I've noticed that out of the 5 or so examples I've tried running, only the official psiturk-example works as expected. First, the server on command fails because these repos don't seem to include the custom.py file in the top level directory (or anywhere?) - copying the custom.py file from psiturk-example fixes this. Second, it seems like there must have been some recent changes in how the html must be formatted (e.g. certain attribute requirements) - I simply edited certain lines from the ad.html and default.html template files to match those from psiturk-example.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/NYUCCL/psiTurk/issues/361?email_source=notifications&email_token=AAM5SQIB5QWNLDVHAZZIUC3QA477TA5CNFSM4IGHGCF2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HA7VTPA , or mute the thread < https://github.com/notifications/unsubscribe-auth/AAM5SQN6L23GYXAQP64MN4DQA477TANCNFSM4IGHGCFQ

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NYUCCL/psiTurk/issues/361?email_source=notifications&email_token=AAI6Y7NBFZZY46UKAV3ZWQDQA5NONA5CNFSM4IGHGCF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2UHCPA#issuecomment-514355516, or mute the thread https://github.com/notifications/unsubscribe-auth/AAI6Y7JJGX6TW7HGHV5CHODQA5NONANCNFSM4IGHGCFQ .

deargle commented 5 years ago

@yang-yangfeng

Second, it seems like there must have been some recent changes in how the html must be formatted (e.g. certain attribute requirements) - I simply edited certain lines from the ad.html and default.html template files to match those from psiturk-example.

Two likely problems here. One, if you are seeing a blank page, then it is your ad blocker blocking the ad because it has an id='ad' specification (facepalm). I renamed that in later template defaults to be id='not-an-ad', lol.

Other problem, older versions of the default.html template did not have a auto-debug-url feature. To debug, you need to use a url like unto psiturk debug -p, that gives you random url values for assignmentid, hitid, workerid. It is impossible to backport templates, when they are designed to be overrideable, unless the owners of all of the repos updated their stuff (I'm not the owner).

First, the server on command fails because these repos don't seem to include the custom.py file in the top level directory (or anywhere?) - copying the custom.py file from psiturk-example fixes this.

I just fixed this. The code tried to handle an unfound custom.py, but it was catching the wrong exception name https://github.com/NYUCCL/psiTurk/blob/1b08baa9eacfb1f2acd2f0977c48b09af4c5f50b/psiturk/experiment.py#L82-L94

I have swapped ImportError for the correct ModuleNotFoundError, thanks for pointing this out.

Besides for these two things, I don't know what else might stop an experiment from running?

@jacob-lee I also added a detection of missing closepopup.html and thanks-mturksubmit.html if use_psiturk_ad_server = false in config.txt, because I also miss adding those sometimes. Server will now refuse to start if they are missing.