Adding Partial-Masking Functionality

[lydiastone16]

We are designing a new experiment where teams can be partially-masked (some pairs of teammates are unmasked to each other, some pairs of teammates are masked to each other).

We will create multiple (theory-driven) mechanisms to decide which pairs to mask vs. unmask. An example mechanism is to have team members complete a survey after a round saying which teammates they liked and didn’t like. Based on these survey results, in a later round, teammate pairs that mutually liked each other are unmasked to each other; while everyone else is masked to each other. Another example mechanism is to mask/unmask teammate pairs based on their interactions in previous rounds. Teammate pairs who appeared to converse directly in previous rounds (e.g., talked in a pair without another member talking) are unmasked to each other; while everyone else is masked to each other.

This experiment requires

1. A data structure that stores: for each pair of teammates, if they are unmasked to each other. We think code changes will be on the backend, though the frontend should also have access
   to this data structure.
2. A way to execute the partial-masking (i.e. the name switch). We think code changes will be on the frontend (front/src/containers/batch.js).
3. Toggles on the admin page There should be toggles on the admin page that either enable 'selective-masking' on particular rounds or leave the default 'full-masking' on.

[deliveryweb]

Hi @lydiastone16! I think it is an interesting feature to implement. Could you give a more detailed description of the algorithms deciding whether the pair will be unmasked? Thanks!

[deliveryweb]

And another small question, imagine we have a pair of user1 and user2. If user1 happens to be unmasked to user2 will the user2 be always unmasked to user1 or there can be situations where we want him to be masked?

[deliveryweb]

And what to do with the situation, when two teammates liked one another and in the next round they are moved to different teams? Maybe it is better to do a global list of pairs for all the users of batch, not a new one for every team? @lydiastone16

[lydiastone16]

Hello @deliveryweb! Regarding the 1st comment, we haven't decided which algorithms to use for deciding whether a pair will be unmasked. But a detailed algorithm of one of the previously mentioned examples is:

Round 1: A team of 4 (all masked) work together.

After Round 1, every team member completes a survey, where they rank each of their teammates on a scale of 1 (strongly disliked teammate) to 7 (strongly liked teammate). We say a pair of teammates "mutually like" each other if they mutually ranked each other a number higher than 4.

Round 2: The same team of 4 (partially-masked) work together again. Every pair of "mutual-like" teammates is unmasked to each other. Everyone else ("mutual-dislike" or "1-way-like) is masked.

Does that help? :)

[lydiastone16]

@deliveryweb Regarding the 2nd comment, If user1 is unmasked to user2, then user2 must be unmasked to user1.

For any pair of users (user1, user2), exactly one of these happens: 1) user1 is unmasked to user2 AND user2 is unmasked to user1 2) user1 is masked to user2 AND user2 is masked to user1

Does that make sense? :)

[lydiastone16]

@deliveryweb Regarding the 3rd comment, If I understand the question correctly, we will only apply partial-masking to 1 round (i.e. 1 team). So we only need to store which pairs to mask vs. unmask from that 1 team.

[markwhiting]

I’d just add that we might want to do it for more than one round in the future but sticking to one for now is ok.

[deliveryweb]

the rounds in addTemplate now have 'selective-masking' option, if selected, the round will be partially masked according to partial masking function. There is only 1 function implemented by now, the one with chat activity, if you come across a better algorithm, we can implement more. @lydiastone16

[lydiastone16]

Great, thanks so much! :) We'll get back to you after we come up with some partial masking algorithms!

[lydiastone16]

Update! We've picked a mechanism to decide which pairs to mask vs. unmask: mutual likes and mutual dislikes.

For context, this is how our new experiment works: Round 1 Team A (4 people) work together on a task [completely masked].

After Round 1 Each member of Team A fills out an individual survey, ranking how much they like each teammate (3 teammates total). The survey, for each teammate, asks the question:

      <Teammate name> is someone I would like to work with again. 
                  - Strongly Disagree
                  - Disagree
                  - Neutral
                  - Agree
                  - Strongly Agree

Round 2 Reconvene Team A to work together on a task [partially masked]. Pairs of teammates that mutually liked each other (i.e. rated each other 'Strongly Agree') are unmasked to each other. Everyone else is masked.

Round 3 Reconvene Team A to work together on a task [partially masked]. Pairs of teammates that mutually disliked each other (i.e. rated each other 'Strongly Disagree') are unmasked to each other. Everyone else is masked.

Important: The partial masking mechanism of Round 2 and Round 3 is randomized. Either Round 2 partially masks based on mutual like pairs + Round 3 partially masks based on mutual dislike pairs; or Round 3 partially masks based on mutual like pairs + Round 2 partially masks based on mutual dislike pairs. This will be decided randomly for each team.

@deliveryweb

[dtatarenkov]

if I understand correctly, you want this to work with multi-team batches as well as single-team ones. I don't really see the way this could work with multi-team batches, because there is specific team-shuffling algorithm and we can't just make teams with the same users in rounds 1, 2, 3. We can implement this for single-team batches. Is it ok? @lydiastone16

[lydiastone16]

Yes, this experiment is for **single-team** batches. Thanks for clarifying! @dtatarenkov

[dtatarenkov]

in dev

[lydiastone16]

Actually, dev is no longer needed! Thank you!