search

Watts-Lab / surveys

Library of surveys for deliberation experiments
MIT License
3 stars 4 forks source link

Where should survey computations be conducted? #13

Closed markwhiting closed 2 years ago

markwhiting commented 2 years ago

We have the situation that many surveys have some computational workflow to go from a response to a value, e.g., CRT needs to be checked against the correct answer, summed and normalized by the number of questions. Different versions of CRT potentially have different aggregations.

Right now we do this in R after the survey is done, but, you could imagine doing something where the surveyjs stores the aggregated result (in addition to the individual question responses in case thats useful for updating things later), so that the aggregation is local to the survey, and that any commonly named aggregated column, e.g., CRT, or political indices are checked at the relevant point.

JamesPHoughton commented 2 years ago

I believe SurveyJS has the ability to do some basic calculations internally, although I'm not sure how much. Here's where it's referenced in the documentation: https://surveyjs.io/Documentation/Library?id=LibraryOverview#conditions-functions @bsmith6202 - would you be up for looking more into what is possible?

markwhiting commented 2 years ago

Ah, thanks, I was assuming it was doable and more wondering if it seemed like a good idea.

I checked and it seems totally doable (either via built-ins or custom functions in more complex cases), but reading the docs makes me think surveyjs is a bit more limited than I thought.

JamesPHoughton commented 2 years ago

I think its a very good idea, because then we have the necessary aggregations/calculations stored with the survey of record. Then if it's used in multiple places, there is no ambiguity about how the results are tallied, and it should be plug-and-play to get the reported score.

markwhiting commented 2 years ago

Agreed. Will resolve this but I think adding this notion to any of the main instruments seems like a good plan. I'm going to make it a requirement for the ones we transition over.