Research: Digest PACT Wizard mural and provide UX guidance on flow, recruiting strategy

[wesrowe]

Description

User story

AS A Researcher I WANT to review and provide feedback on the wizard flow from the perspective of a researcher and research participant SO THAT we can identify options and tradeoffs around prototyping for research.

Engineering notes / background

Danielle Thierry's PACT Wizard mural.

Analytics considerations

Quality / testing notes

Acceptance criteria

• [x] Reviewed mural and formulate recommendations
• [ ] Hypothesized how recruiting strategy could reduce need for branching logic
• [ ] Met with Danielle about feedback and next steps toward prototyping, recruiting, research planning

[wesrowe]

Meeting set for 6/14: Danielle, Cindy, Wes

[cindymerrill]

@wesrowe I reviewed the Mural closely, and I made an outline of all the survey logic we need in order to avoid asking users anything that we can figure out the answer to (see outline below). I also added comments to Danielle in the Mural about survey logic conditions that didn't appear correct. It doesn't seem like prototyping this with multiple surveys in Optimal Workshop is going to work because there are so many checks when branching and when deciding to ask specific questions to see if in the user's years of service. And if we don't do this, then we'll be asking users extra questions that they shouldn't have to answer, resulting in a poor assessment of the total number of questions presented.

Survey Flow w/Logic:

• ask for service years and whether served outside US
• if served outside US
  • if service includes any time 2001-present, ask re location after a date
  • if service includes any time 1990-present, ask re location after a date
  • if service includes 1962-1975, ask re location and date range (SWAP w/next Q?)
    • if N and service includes 1962-1980, ask re another location and date range
      • if N and service includes 1962-1976, ask re another location and date range
        • if N and service includes 1972-1977, ask re another location and date range
          • if N and service includes 1967-1971, ask re another location and date range
            • if N and service includes 1969, ask re another location and date range
  • if service includes 1977-1980, ask re location and date range
  • if service includes 1966-1967, ask re location and date range
  • if service includes 1968, ask re location and date range
• if didn't serve outside UX and service includes 1962-1986
  • ask 3 q's; stop if get Y
• if service includes 1953-1987
  • ask 1 q

[cindymerrill]

Are we going to test/prototype the questions about health conditions, too? Are we going to test/prototype the results screens? @wesrowe

[cindymerrill]

@wesrowe Want to talk about this before our meeting with Danielle next week?

• I still recommend prototyping with a survey tool that has the ability to branch based on date ranges (not Optimal Workshop or Microsoft Forms). If we use CodePen instead, it might be a LOT of tedious coding for an engineer.
• We could recruit by service dates, but it's quite possible that someone with a long time in the military could have service dates that overlap multiple date ranges. And there are a lot of overlapping date ranges across all of the questions.

[cindymerrill]

@wesrowe I scheduled a meeting with you on Tuesday, and just scheduled another one with you on Monday because as a result of conversations I had with two other Ad Hoc researchers, I'm thinking that finding out more may help us decide how to prototype. I have questions for us to discuss in the new first section of my research brainstorming doc: Questions to answer to inform prototyping options.

The questions are about research outcomes and prototype requirements, including the following:

• What are we trying to learn from this research?
• What are potential outcomes of this research?
• How important is it that participants interact directly with the prototype to see the question and select a response?
• How important is it that the prototype be accessible to screen reader users? Since there’s no front end to test, I would think we could wait until later.

[cindymerrill]

Health conditions will NOT be covered in MVP, according to Danielle

[cindymerrill]

Danielle updated her Mural with questions, and so I've updated my outline of the survey flow accordingly:

This can be seen in my research brainstorming doc.

[cindymerrill]

And here's a quick summary of the service years covered by Danielle's questions:

This can be seen in my PACT Act Wizard years in questions spreadsheet

[cindymerrill]

FYI @wesrowe See my previous two comments with updated summarizes of Danielle's revised questions

[cindymerrill]

FYI @randimays See my previous two comments with what I shared during today's meeting

[cindymerrill]

Obtained an Optimal Workshop license and signed in to test survey branching logic. Confirmed that branching can happen based on single-select questions (radio buttons) but not multi-select (checkboxes); sent a chat to customer support to confirm--CONFIRMED 6/15/2023.

So, we're unable to make the service years question into multi-select checkboxes and then branch from there to appropriate question flows. We CAN branch if the time/war options are single-select radio buttons, but then would need to repeat the survey for each time/war checked. @wesrowe @jilladams

[cindymerrill]

@wesrowe and I met with Danielle yesterday. She showed us a new version of the content Mural in which specific dates were removed from all the yes/no questions; she and Martha thought it would be more accessible to Veterans, and I heartily agreed. Danielle still needs to get approval from the SME's.

We heard Danielle's ideas for potential research goals, which included the following:

• Find out if there are too many questions
• Find out if the individual questions are too complex to answer
• Find out if we can deliver results that make sense--i.e., actually help people and not confuse them, especially if the specific month/day date ranges are included in the results

We talked only a little about prototyping the wizard, and Danielle pointed out that the Mural wasn't intended to be a complete design (e.g., the yes/no questions might need an "I don't know" option). We talked about the possibility of prototyping using Optimal Workshop, which may result in showing more questions than are strictly necessary because the tool's branching logic capability is weak, and may require creating multiple surveys to choose from, depending on which war(s) the Veteran participant served in.

We should wait until the content gets approved before jumping into research planning. Currently our tentative recommendation is to prototype using Optimal Workshop. Next week we'll find out more about what's involved to prototype using CodePen, which is the other option being considered.

We discussed how recruiting could help make a survey tool (optimal workshop) more viable for prototyping

• Careful recruiting could yield participants whose service falls into just one or two eras (question groups)
• We're okay with the researcher providing a participant more than one survey link if their service spanned eras
• If we want to test a checkbox type question, OW may not be able to use that input for branching.

Next steps:

• DT to run new date-later approach by stakeholders for approval
• we should do an OW proof-of-concept to confirm
• When content is final/approved, we can write research plan

FYI @jilladams

[cindymerrill]

Today's scheduled CodePen meeting was postponed until next Monday. This meeting is input to making the prototyping decision. FYI @wesrowe @jilladams

[wesrowe]

@cindymerrill, I thnk that meeting was more relevant to another PACT ticket, #13722 or #14033. I would close this one.