👥 meeting March 29

[JamesPHoughton]

In our experiment yesterday, we had a server restart that threw everyone out of the game. Many of the participants were able to rejoin, and we were able to get data.

• The bug is tracked here: https://github.com/empiricaly/empirica/issues/269
• Remediation steps are here: https://github.com/Watts-Lab/deliberation-empirica/issues/484

[xehu]

Notes:

• There was a server issue during the last pilot, and several participants needed to rejoin (but they were re-randomized into different conditions, so there should be some kind of asterisk around the data).
• IDEA: Potentially add some kind of 'notes' field in order to log issues that don't appear in the JSON data? [Question for @JamesPHoughton - do we know that ALL participants are affected, or only some subset of them?] Answer: Our assumption is that everyone in the data was in the game for about 20 minutes, talking with a different team.
• Will postpone tomorrow's pilot until ~April 11 or so, so that we can ensure the server issues are fixed.

[xehu]

Feedback from Will's presentation:

• How do zoom meetings differ from in person meetings?
• The leadership treatment is bundled with a bunch of other treatments

  the leader treatment bundles together extra speaking time with agenda/norm-setting and the implicit status of being called a “leader” (although this status is arguably also muted by the fact that people know the leader was randomly-selected).  My response: yes, we wanted the strongest possible treatment for our initial studies, since we want to be able to manipulate influence as much as possible, but future studies can unbundle the treatment to the extent we’re interested in its constituent mechanisms.

Response to zoom v. in person (from James):

> R2: what I do find myself wondering is about the external validity of what will be learned from these studies. From my read of the proposal, it seems that all of the studies will be conducted in an online, Zoom-type environment.  But is that where most people have conversations about politics? I suspect today that much more political discussion is either taking place face-to-face or asynchronously on social media.  So what are we actually going to be learning once all of the many studies proposed here across the experimental space have been carried out?  Is it simply a set of best guidelines for NGOs that want to offer people the opportunity to come together to discuss politics in a civil setting in virtual video chats? Or does the team aspire to make more general contributions about means for dialing down political polarization in the country writ large? If the latter, what kind of lessons based on what is learned in the experimental studies could be applied outside of these highly controlled video interactions?
External validity is front-of-mind for us as we design both the participant interface and our approach to generalization, and these efforts correspond to two reasons we believe this design can make useful contributions to real-world deliberation.
The first is that the distance between the lab and the field is shrinking. Trends in technology and remote work are increasing the number of real-world deliberations held online, and in some ways, the world is moving toward conditions that we can replicate in a virtual lab. As this project matures, we plan to invite community groups to use our tools to facilitate deliberations on their own topics, bringing the lab fully into the field. Additionally, a number of bridging organizations are already working to move a portion of their in-person workshops online, with the goal of increasing scope and scale. One example is AllSides Connect (https://www.allsides.com/connect), a project which brings people together through facilitated online video conversations to talk about political topics.
We expect in the future to incorporate other media of exchange, for example, text chat or simulated social media interfaces. Even in the physical lab or field setting, we expect to be able to draw heavily on our infrastructure investments to serve interventions. For example, showing participants a display on their smartphone which can indicate their level of participation.
The second reason we are optimistic about the usefulness of these experiments is that they provide contextualized priors for experiments we may wish to conduct in a physical context. Just as we use existing data to identify the most informative next in our virtual lab, we can use the data we collect to identify high-leverage experiments to conduct in a physical lab, making this more expensive form of experimentation yield more scientific value per sample.
Importantly, while our focus is initially on political discussion, we hope to use this tool to learn about the everyday deliberations that go on within companies or communities and provide support for the full range of deliberative exercises. 

[xehu]

Analyses:

• Videos are in Amazon S3 (folder for each discussion and within folder, an audio/video file for each participant) -- some stitching required if you disconnect and reconnect; Videos are time-synced across all participants in the same room
• Treating each audio stream as a separate transcription is fine (we want to know how much people are awkwardly pausing, speaking over each other, etc).
• Want to start making progress on audio/video/transcript analysis
• Initially, will need something like a 'manipulation check' (confirm that the leader speaks the most)
• Three levels of processing: (1) transcript; (2) audio features, and alignment of transcript and audio features; (3) rolling these up into an analysis at the utterance level (start time / stop time, etc.)

[willschulz]

Survey addition: add a 7-point ideology on a likert scale from very liberal to very conservative