Potential bug in survey plugin `accuracy`

[jodeleeuw]

Just discovered some data that looks like this:

"response": {
      "P0_Q0": "Murder",
      "P0_Q1": "Emily French",
      "P0_Q2": "Party",
      "P0_Q3": "Wealth"
    },
    "accuracy": [
      { "P0_Q0": true },
      { "P0_Q1": true },
      { "P0_Q2": false },
      { "P0_Q2": false },
      { "P0_Q3": false }
    ],

There are duplicate entries for P0_Q2. Maybe this is intentional behavior if a subject changes their response? It created some difficult post-processing in R because we could no longer unnest() the accuracy column to match response to accuracy.

[becky-gilbert]

Thanks for the feedback Josh. No this isn't the intended behavior. The data should just store the final response.

The response scoring function is called whenever the value is changed, so I think you're right that the duplication comes from the participant changing their response. And it looks like I already made a note about why using the survey's onValueChanged event is problematic for other reasons 😬 https://github.com/jspsych/jsPsych/blob/5cf4e5bc093749b0dd8601f9fdc5373447dca3b4/packages/plugin-survey/src/index.ts#L483-L486 I think the problem was that I couldn't find another question-level event to hook into. The survey events can be found in the docs here.

I'm also not sure why "accuracy" is an array of objects in the data, rather than a single object like "response". If it were an object, then updating the accuracy when the question value changes would cause it to overwrite rather than duplicate, since the questions names would be unique keys.

(Just writing this out here in case it's urgent and you need to fix it ASAP, since I won't have a chance to fix this until next week.)