Watts-Lab / surveys

Library of surveys for deliberation experiments
MIT License
3 stars 4 forks source link

Implement team viability measure #12

Closed JamesPHoughton closed 2 years ago

JamesPHoughton commented 2 years ago

One of the most important measures after a deliberation is the participant's subjective experience of working with the other participants. One survey that implements this is:

Cooperstein, Jessica Nicole. 2017. “Initial Development of a Team Viability Measure.” Edited by Suzanne T. Bell. Master of Arts, DePaul University. https://via.library.depaul.edu/cgi/viewcontent.cgi?article=1211&context=csh_etd.

There are a number of questions in this survey that all try to get at the same fundamental underlying concept, and we will collapse those measures into a single outcome variable. This may require us to write an aggregation function in the .json file that defines the survey, and modify the surveyWrapper to call this function before storing the data. See discussion here: https://github.com/Watts-Lab/surveys/issues/28

Ready to close when

sky1113 commented 2 years ago
JamesPHoughton commented 2 years ago

@markwhiting - when you have used this survey in the past, have you used all 20 questions in the paper, or have you used a subset?

markwhiting commented 2 years ago

I used the 14 question version in appendix E of the paper, which is also here → https://github.com/StanfordHCI/bang/blob/master/txt/midsurvey-q.txt

One of my students showed that we could collapse this into one question (the first one if I remember correctly), but we didn't have enough data to validate so we have not published that.

JamesPHoughton commented 2 years ago

How did you go about figuring out which was the one question to ask? Not sure how one goes about doing this sort of thing. My ideal would probably be about 3-5 questions, I think, if we knew which ones.

Do you collapse the questions into a single measure, or use them as independent measures?

markwhiting commented 2 years ago

We collapsed into a single measure, as is standard with a Likert of this type. Let me see if I can get any helpful insight from the student who worked on the simplification bit.

markwhiting commented 2 years ago

My thoughtful student helped me realize that we'd actually included some initial analysis on this scale in a past paper → see pp 12–13 here for a few paragraphs on this.

We started a validation process but it's not done yet. Let me know if someone here wants to take that on as a reasonably straightforward paper and I loop in the appropriate people.

JamesPHoughton commented 2 years ago

@sky1113 - lets implement the 14-item for our experiment this week.

@markwhiting It would be great to do the full validation - let's outline it as a project for the fall.

markwhiting commented 2 years ago

Yep, sounds great! I'd say it's a rather good independent study project.

JamesPHoughton commented 2 years ago

Adding an issue for tasks for the fall, I expect this will become its own project eventually: https://github.com/Watts-Lab/deliberation-project/issues/38

kepstein23 commented 2 years ago

Blocked on #28

JamesPHoughton commented 2 years ago

Lets split this into sub-issues: