Live Debug?

[williamstome]

I just launched a HIT for a small number (thankfully) of participants. I then got a bunch of emails from folks saying they got to the end of the HIT and there was now way to submit. I looked at the logs and it looks like the HIT was somehow both live on the PsiTurk servers AND in Debug mode? All participants' HIT IDs and Worker IDs are showing up as debugXXXX.

Any idea how this could happen?

I'm running PsiTurk 2.3.0.

Here is my Config file:

[HIT Configuration] title = REDACTED description = REDACTED amt_keywords = REDACTED lifetime = 24 us_only = true approve_requirement = 80 contact_email_on_error = REDACTED ad_group = REDACTED psiturk_keywords = REDACTED organization_name = REDACTED browser_exclude_rule = MSIE, mobile, tablet allow_repeats = false

[Database Parameters] database_url = postgres://iREDACTED@ec2-REDACTED.compute-1.amazonaws.com:5432/REDACTED table_name = hri014

[Server Parameters] host = 0.0.0.0 port = 22362 cutoff_time = 60 logfile = server.log loglevel = 2 debug = false threads = 1

[Task Parameters] experiment_code_version = 6.0 num_conds = 16 num_counters = 1

[Shell Parameters] launch_in_sandbox_mode = true

use_psiturk_ad_server = false ad_location = https://REDACTED.herokuapp.com/pub

[deargle]

Does it work for you when you post a sandbox hit?

All participants' HIT IDs and Worker IDs are showing up as debugXXXX

Are you sure? This is what any of your testing runs would have shown up as.

The only way I can think of that would lead to legitimate workers getting into your hit with debug ids is if they visited the "default" route, which by default is accessible at the base url, https://REDACTED.herokuapp.com/ . If your /pub route is serving the standard /ad route, then there is logic in that file which picks up the workerid and assignmentid from the url to pass through for participants. Did you do anything drastic to your ad.html template?

[williamstome]

Yeah, I'm pretty sure -- the timestamps are all for after the experiment was launched, and the ages reported from participants look like normal ages and not the ages I put in while testing (e.g., 99).

Re: the ad, nothing is changed beyond filling in the placeholder XXXX's.

[williamstome]

Sandbox results maybe help elucidate problem? When I find it in the sandbox, the url is https://workersandbox.mturk.com/projects/33EWWXEH81CF0XD23PZ81MR19Q7043/tasks/3PKVGQTFIHYITYDA44WJQOAGIN8YRC?assignment_id=3C44YUNSI23W5UZYLIAIXSVG28UDPS&auto_accept=true

But when I then click on "begin experiment" the popup window has the URL https://agile-spire-06406.herokuapp.com/exp?hitId=debugJOIyG&assignmentId=debugbW7wh&workerId=debugRYa0Q&mode=debug

[williamstome]

I'm guessing once the experiment starts, the workerid / assignmentid should be the same as in the sandbox url?

[deargle]

I just tested your sandbox url. The first page I see is your default.html template, which, as I said before, generates debug URLs. I see that you specify /pub route in your config you shared, but that is not being used. Any reason to not upgrade to latest psiturk? Not that I know of any such bug in older versions, but would rule that out.

On Fri, Jun 26, 2020, 3:44 PM Tom Williams notifications@github.com wrote:

I'm guessing once the experiment starts, the workerid / assignmentid should be the same as in the sandbox url?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/NYUCCL/psiTurk/issues/429#issuecomment-650416802, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI6Y7LFGAWHCPOE6TNGUSDRYUJCDANCNFSM4OJRHJUA .

[williamstome]

Huh, that's really bizarre. It's definitely in the config file, and I've definitely pushed that version to Heroku. I will try updating to latest PsiTurk and see if that fixes this.

[deargle]

That wouldn't be heroku's fault-- that config setting is passed to mturk when registering the hit. If you later change it and create new hits without expiring the old ones, there will still exist on mturk hits with the old ad url. Only once those got used up would the ones with the correct ad url be available to workers. Or, mturk would create two different listings for your hit - one for each version of the ad url. The thing to do is expire all old ones.

On Fri, Jun 26, 2020, 5:53 PM Tom Williams notifications@github.com wrote:

Huh, that's really bizarre. It's definitely in the config file, and I've definitely pushed that version to Heroku. I will try updating to latest PsiTurk and see if that fixes this.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/NYUCCL/psiTurk/issues/429#issuecomment-650454919, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI6Y7JBOPUPA345RNFVDC3RYUYIXANCNFSM4OJRHJUA .

[williamstome]

1. I updated psiTurk and the error above was fixed.
2. HOWEVER, I think you are completely right, and in fact the issue may have been that "/pub" was not in the URL when the hit was first created, and then was not updated correctly (i.e., we extended rather than expiring and re-creating). Thanks for all your help and sorry for the bother.