'curTheta' error in CAT CDI

[JMankewitz]

Hello! We had a participant who got the attached error while filling out the CAT CDI.

Here's the message from the participant: "There was a key error note during the first questionnaire, where it asked if my child could say "make". I couldn't finish but I wanted to."

And here's the participant's link that's now broken: https://webcdi-dev.stanford.edu/form/cat/fill/fd4503e11d14b7849b4af89a1e395803aedf8552a41fac533a60682a22795fdc/

[HenryMehta]

@JMankewitz @vmarchman

I am unable to replicate this error in order to work out how they did it so I can prevent it.

I can see the problem appears to be that they are being asked to repeatedly respond to the same item. This is some of the data assigned to the record:

administered_items | [206, 344, 344, 344, 344, 344] administered_responses | [True, True, True, True, True, True] administered_words | ['find', 'make', 'make', 'make', 'make', 'make']

As you can see they've been asked (or rather they have answered) 'make' 4 times. This is causing an error to come back for the CAT API which results in this error. The only way I can think of the repeated words happening is the user clicking the 'Yes' button multiple times. I've tried doing that but the system is handling it.

Can you ask the user:

1. if they were asked 'make' multiple times.
2. the browser they were using
3. the system they were using (were they on a phone, tablet or PC)
4. Could they recall, describe everything that happened (eg multiple clicks or screen refresh or anything they can think of)

Thanks

[HenryMehta]

@JMankewitz @vmarchman No need to ask. Found the issue. It is the use of F5 (screen refresh) and resubmitting at the same time that causes the issue. Either by themselves is fine but the combination causes this error

[HenryMehta]

@JMankewitz @vmarchman

I think I have now resolved this. I've added code so that the page cannot be freshed. This removes the issue.

For the item above, I've gone into the database and removed the repeated words/responses and I've check and it now goes to the next word so the user can complete the form if required.

[JMankewitz]

Hello! It seems we have another participant who got a similar curTheta error, unfortunately. This time its subject 10 in the Computer Adaptive Test

Here is this participants link with the error: https://webcdi-dev.stanford.edu/form/cat/fill/40d9c42115c00c9896ad34cc786d5c94356351224f6bfd6c467153593cce2e22/

This participant said they werent refreshing at the time. Full message from the participant is

Hi - I am attempting to complete the first survey in this string and, after attempting to report that "No" my child cannot say the word ear, I am being sent to a page stating "KeyError at /form/cat/fill/40d9c42115c00c9896ad34cc786d5c94356351224f6bfd6c467153593cce2e22/"

This is followed a lot of coding that doesn't mean anything to me. My husband works in IT and says that it appears to be on Stanford's end. How do I proceed?

[HenryMehta]

@JMankewitz I've fixed the data so that the survey will now proceed. I need to understand the steps the participant took in order to find the error because I cannot work out how it was created because the only way I've been able to create it is how I described, which is now fixed. I will spend some time today trying other approaches to create it but I cannot think what might cause it.

[JMankewitz]

I reached out to the participant and they don't remember doing anything strange. They weren't refreshing, just submitting the page. They were just trying to submit their response "No for 'Ear'" if that helps at all...

It seems to only be happening when there are multiple people filling out administrations at the same time (none of us have been able to replicate it). Could that be related?

[HenryMehta]

@JMankewitz That's a possibility and a big issue if it is the problem. I cannot complete two surveys at the same time. Could you trial it with someone local to you please, attempting to press enter at roughly the same time and seeing if you get the error Thanks

[JMankewitz]

I got more information from the participant:

Hi - I don't know that I was doing anything in particular when it crashed. I was taking the survey in Chrome, and clicked on the "no" option for the first question. I had several other tabs open in my browser, but wasn't using any of them at that time. The only thing I can think of that might have made something break is that my mouse has been having intermittent issues recently where sometimes when I click on something, it sends a double click into the program very rapidly. I clicked "no" and nothing happened after about 10 seconds, so I tried clicking no again, and then flipped back to the Prolific hub to see if I could even open access another tab or whether the computer had frozen, and when I came back to the survey page, I had the Key Error page with all of the coding. I tried to go back in my browser window at that point, but it went back several frames to the screen requesting some demographic/background information, not to the question about whether or not my child could say "ear," and re-submitting that just took me back to the Key Error page.

I could see something strange happening with someone submitting twice due to lag - I'll try testing with someone also submitting locally, but I'll also test on a throttled network browser and see if I can replicate.

[JMankewitz]

I've been able to replicate on Chrome Version 86.0.4240.111 (Official Build) (x86_64) w/ slow 3G network throttling.

Specific behavior: Click "yes" or "no", click again while page is reloading/sending data - this causes the 'curTheta' error

This does not break on Firefox even with network throttling, haven't tested on Safari.

[HenryMehta]

@JMankewitz Great spot. I tested in Firefox and it was fine - didn't think to try another.

I've amended the code so as soon as a response is entered the screen goes blank. This prevents double tap. I've deployed to dev. Is this ok?

[JMankewitz]

Seems like a fine stopgap & works as expected.