[CAIA a11y Research]: IIR, Welcome Pathway - Research Guide and Convo Guide Feedback (October 2024)

[!NOTE] QUICK SUMMARY: CAIA a11ys will review IIR's Welcome Pathway research plan and conversation guide, and provide feedback in this ticket.

Issue Description

... help review the AT research plan and conversation guide I am putting together. I created a ticket thinking this would help me get paired with a specialist ...

Toggle to view the problem statement, desired outcomes and solution approach as of Sept. 17, 2024.

### Problem Statement [c/o Product Outline / Problem Statement Section](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/veteran-onboarding/product-outline.md#problem-statement) > _VA.gov needs to have a defined onboarding experience on VA.gov. We hypothesize that a lack of digital onboarding guidance for newly transitioning Veterans presents challenges in efficiently and effectively getting Veterans the relevant services and resources they need, leading to potential frustrations, delays, and missed opportunities for support and access to eligible benefits._ ### Desired Outcomes [c/o Product Outline / Desired Outcomes Section](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/veteran-onboarding/product-outline.md#desired-user-outcomes) > - _Basic understanding of challenges associated with the transition process, specifically with regard to how VA.gov could remedy those pain points_ > - _A simplified Onboarding experience that walks a new Veteran user through key activities and benefits they should do to get started._ ### Solution Approach [c/o Product Outline / Solution Approach Section](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/veteran-onboarding/product-outline.md#solution-approach) > **After digesting the research and looking at the comparative analysis, we are tasked with finding ways to** > > 1. incorporate an onboarding experience within an existing product and > 2. overlaying the experience in a temporary manner so that users who do not need the onboarding experience can dismiss it. > > _The initial phase will be rolled out to welcome Veterans who have not created a VA.gov account and prompt them to front-load data that VA.gov can then utilize when the Veteran is going through the existing benefit application experience. This provides a complementary support structure to the existing VA experience.gov while also utilizing a step-by-step type approach for those who might feel overwhelmed or as though they are not entitled to all the benefits offered on VA.gov._

About the Team

Team name: IIR (Iterate, Innovate, Run)
OCTO-DE product owner: Jeff Barns
Product manager: Pete Egan
Designer: Liza McRuer/Raquel Eisele
FE engineer: John Rodriguez/Nick Sayre
Product/team Slack channel: #iir-product-teams-public

Supporting Artifacts

DRAFT AT Convo Guide (GH)
DRAFT AT Research Plan (GH)

### a11y Tasks
- [x] Collaborate with the team to understand more background information about the project 9/25/2024
- [x] Review the Convo Guide - Jamie 9/25/2024 [refer to convo guide comment in this ticket 93662](https://github.com/department-of-veterans-affairs/va.gov-team/issues/93662#issuecomment-2376838644)
- [x] Review the Research Guide - Evan Started 9/25/2024 [refer to research plan comment in this ticket 93662](https://github.com/department-of-veterans-affairs/va.gov-team/issues/93662#issuecomment-2376851207)
- [ ] Complete Research Plan Review - Evan/Jamie
- [ ] Identify and provide feedback on the Convo Guide in the comments of this ticket
- [ ] Identify and provide feedback on the Research Guide in the comments of this ticket

### Acceptance Criteria
- [ ] IIR receives feedback on the convo guide - Provided 9/25/2024 c/o Jamie - [refer to convo guide comment in this ticket 93662](https://github.com/department-of-veterans-affairs/va.gov-team/issues/93662#issuecomment-2376838644)
- [ ] IIR receives initial feedback on the research guide

## Ticket Updates (CAIA Internal)
- [x] _Connect to an `Epic` or `Intake` (what body of work is this a part of?)_
- [x] _Label with `Originator/Team` (product team or stakeholder requesting support)_
- [x] _Label date in the `Open Date` field_
- [x] _Label with `Estimate` (level of effort expected for this ticket)_
- [x] _Add `Assignee(s)` name(s) to ticket_ (include Naomi)
- [x] _Add `Assignee(s) name(s) to each task` they will complete via handle tag (if known)_
- [x] _Select a `Priority Level`_
- [x] _Update date in `Last Checked` field_
- [ ] _Label with `Actual` (level of effort it took to complete this ticket)_
- [ ] _Update date in_ `Closed Date`
- [ ] Mark any tasks in the original research ticket as done ☑

### Related Tickets
- [ ] #92823

Convo Guide Review

c/o Jamie 9/25/2024 post meeting with IIR

Suggestions for the convo guide:

Ask whether, and what, assistive tech they're using during the Intro. Recommend not waiting until the setup portion because it's fairly common to get either "false positive" signups - folks who don't actually use AT - or participants that signed up with one AT, but prefer to use another (ex., signed up with JAWS, but using an iPad). Make sure they're using AT first, and disqualify them if not - that way, you aren't wasting anyone's time.
Give options for how to receive the link to the prototype. Zoom chat can be cumbersome to access, so some folks prefer email or dictation.
Create a shortlink. And use a "human" slug - bit.ly/TheTest, for example. This'll make it easier to dictate if needed.
Emphasize that they don't have to log in. Logging in is a huge blocker for people. Make sure it's clear that you're emulating it, and that they aren't actually logging in to anything. It'll be a relief. 😄
Tell folks not to click ahead. If you want them to talk through a screen before clicking on things, say that up front. This is especially important with screen reader users, because screen readers are noisy - if they move ahead and you tell them to stop, they might not hear you over the screen reader.

Research Plan Review

c/o Evan 9/25/2024 post meeting with IIR

Initial Suggestions for the Research Plan

The research dates aren’t reflective of the projected time to conduct sessions.
- Sounds like the prototype wont be developed until end of October at the earliest, so it would be great if the dates can be updated or TBD for the time being.
Looks like 8 total participants are being requested, but only 7 have designated AT (2 desktop SR, 2 mobile SR, 2 screen mag, 1 speech recog).
- Not sure we are doing 2 speech recognition participants, but please update when you have a chance.

Feedback

I had a chance to review the research plan and conversation guide and have a couple pieces of feedback:

Duration

The research plan states that the length of sessions is 2 hours but the conversation guide states that the session will take no more than 90 minutes
- I recommend nailing down how long the sessions will be prior to sending recruitment criteria to Perigean so participants can plan accordingly
- You mentioned this is your first AT study so for reference the AT sessions I've provided support for are typically 90 minutes long. If participants aren't going through the login experience as part of the session I think this should be a good amount of time!

Screener

Consider including screener questions that ask potential participants about their specific assistive tech that they are going to use during the session
- A couple recent research studies that have included AT screener questions in their research plans are:
  - August 2024: Medical Records usability study
  - September 2024: Ask VA dashboard usability study

Recruitment Feedback

Recruit beginners and advanced AT users during recruitment for a mix of perspectives - their needs and behavior may differ.
Grouping sessions into cohorts can help recruiting and with running the sessions.

See an example of how to note schedule and availability

Rule of Thumb: Session Duration

Plan for 2x the time you'd usually need.

The VA Platform recommends 2 hours for most studies. There may be technical difficulties, the need for breaks, or unexpected situations during the study. Beginner AT users will also need more time to complete tasks.

Most sessions will not take this long, as Riley mentions above, so you can always mention that to the user at the start of the session, if it ends up being scheduled for the full two hours.

See an example of how to note session length

Screener Questions

Write assistive technology-specific screener questions

When planning your conversation guide, it can be a good idea to set up the screen sharing first after a few qualifying questions. This way, if you can't get the screen sharing working, you can choose to pivot the session. You could reschedule, cancel or pivot the session to suit your needs.

Confirm Technology

You could ask Perigean to confirm the AT being used with the user, before connecting the user with the call.

This is to avoid the false-positives that Jamie was mentioning in her comment above.

See screener questions example

Additional Information

Research with assistive technology users

department-of-veterans-affairs / va.gov-team