[Research] Payment Info V2: Create UAT Plan and Conversation Guide

[ericasauv09]

Background

We need to prepare UAT research documents for the Payments and Debts V2 initiative. This includes creating the research plan and conversation guide.

Tasks

• [x] Create UAT research plan
• [x] Create UAT conversation guide

Note: there is a folder in GH already created for payment info UAT materials: UAT folder

Validation

• [x] Send links to team to let them review asynchronously
• [x] Have another designer review research plan and conversation guide

[aagosto90]

Draft of research plan and conversation guide are ready for review! Please take a look when you can but no rush since we're carrying this ticket into next sprint and I will get another designer or two to review as well. @Samara-Strauss @ajakabcin @ericasauv09

[Samara-Strauss]

Research plan

My usual feedback with recruiting: While I understand we want to make sure testing is done with representative populations, and that all research isn't just done with 55+ white dudes, it feels like some of the demographic requests are arbitrary, especially given that I can't think of meaningful reasons why they would affect the main thing we are testing -- whether our code works or not.

So, these are my suggestions for how to loosen up recruiting requirements:

• I'd remove the LGBTQ requirement altogether. Remember, this is extraordinarily sensitive information for people to disclose. We should only be asking for this if it has a meaningful impact on what we are testing (ie. the new preferred name and gender identity fields we just added to the profile). Otherwise, I don't think we should ever make this a requirement, especially for UAT where we are mostly testing our code, not our users.
• The race breakdown feels arbitrary to me. Again, I realize we don't want 12 white dudes showing up to any of our research sessions, but instead of plugging in random numbers to fields, you could put something like "no more than 6" in the "white" field, which will guarantee we'll get a more diverse sample without making arbitrary specifications about racial breakdown. Again, if there are specific and clear reasons to talk to certain numbers of people in certain groups, great, but it doesn't feel like that reasoning exists here.

And one other thing not related to demographics:

• I see that you specified in the screener that we need people with education/comp & pen payments OR with debts, but then under "Benefits", those fields aren't selected at all. Do you think this will cause confusion for Perigean? I want to make sure that they don't miss the payments/debts requirement, as that is the single most important recruiting criteria we have.

Convo guide

Looks good to me!

[ericasauv09]

Everything looks great to me, thanks Angela!

[aagosto90]

Thank you so much for reviewing @Samara-Strauss and @ericasauv09 ! I made some updates based on Samara's feedback. I removed the LGBTQ+ requirements because while I'd like to hear from more Veterans in that category, it's fair to say there's no other reason to target that group in this study than general inclusion. I scaled back the other diversity requirements but kept a few race requests - 2 Black, 2 Asian, 2 Latinx, the rest various. I haven't heard from a single Asian or Latinx person in a study so far so I think that's a good diversity requirement to target in this study but I think it's more important to try to recruit people who use assistive tech even if they happen to be white men so hopefully that has scaled it back enough for Perigean to meet our reqs, but if not, I'll adjust that criteria further.

As for the benefits, I'm hesitant to outline how many people I want to have each type of benefit which is why I just put all 12 as "Various" types of benefits. It doesn't matter to me as long as we get enough people who receive benefit payments and debts so the screener question is more important here. I share your concern Samara that they might not notice it with all this additional criteria that's been added so when I submit I'll tell the research ops team that it's our first try using the recruitment tables and the most important criteria for us is still in our screener questions.

[Samara-Strauss]

All of this sounds good to me. Thank you @aagosto90!

when I submit I'll tell the research ops team that it's our first try using the recruitment tables and the most important criteria for us is still in our screener questions

Totally agreed. I know the recruitment tables are new, so let's just keep an eye on this. If we feel like we don't get the right folks for the research because screener questions were overlooked, this would be good feedback to share with the research team.