[CAIA A11y Research - PILOT]: Accredited Representative Management (ARM), Find a Representative

[sara-amanda]

CAIA A11y Support

[!IMPORTANT]

Pilot Prep

• Pilot Date(s):
  • Pilot 1: placeholder session scheduled with @coforma-jamie 3/25/2024 10 a.m. ET
  • Pilot 2: placeholder session scheduled with @eli-oat 3/26/2024 12 p.m. ET
• Assistive Tech Being Used: Screen Readers & Magnifiers
  • Specific types of AT unknown at this time
    • However, @eli-oat and @coforma-jamie you can feel free to check @jfinnerty13 's Google Sheet that will be updated with the AT as sessions are added to the study.
    • Eli is planning: Phone+iOS and Jamie is planning: Windows+JAWS
• Testing Environment: Live website 3/5/2024
• Research artifacts: Available

CAIA Assignee(s)

• @eli-oat
• @coforma-jamie

Description

[!TIP] Running two pilots, instead of one can generate more robust feedback for desktop and mobile accessibility, prior to the study.

A CAIA a11y will participate in pilot(s) for the ARM Team's Find a Representative research study.

• Researcher: The VFS team researcher will test their prototype and conversation guide by conducting the pilot session with a CAIA A11Y.
• User: The CAIA A11Y team member will act as a real user in the pilot session.
• Assistive Tech: The CAIA A11Y team member will follow the prompts given by the VFS Researcher, while using and demonstrating how the behavior of a screen reader on desktop and screen magnifier on desktop would behave.

Assistive Tech Options

• Available AT: Screen readers on a desktop device is currently the primary source of AT that CAIA A11ys are able to support, in addition to screen magnification for user research sessions.
• Additional Specifications: If there is a particular test user, browser or operating system you would like us to use, please let us know in advance via this ticket's comments, tagging the assignee - or reach out to @coforma-terry in the CAIA Slack Channel: #sitewide-content-accessibility-ia

Pilot Testing Options

Type	Session	Time Slots (ET)	CAIA A11ys Available	Pilot Tech Options Available
PILOT 1	3/25	10 a.m. ET	@coforma-jamie	VoiceOver + MacOS, JAWS + Windows, NVDA + Windows
PILOT 2	3/26	12 p.m. ET	@eli-oat	VoiceOver on iOS, macOS, Android TalkBack or JAWS on Windows

About the Team

Toggle to view information about the team

- Team name: Accredited Representative Management (ARM) - OCTO-DE product owner: Jennifer Bertsch - Product manager: Lindsay Li-Smith - Researcher: Janelle Finnerty - Designer: Michael Tri - FE engineer: Holden Hinkle, Colin O'Sullivan - Product/team Slack channel: #benefits-representation-management

Pilot Session

[!CAUTION] Missing Items: We'll be testing on the live VA.gov website._ LIVE 3/5/2024 (PER 2/29 MEETING)

• Product outline
• Research plan
• Research conversation guide - Available as of 3/7/2024
• LIVE URLS
  • "Get help from a VA accredited representative or VSO" (Landing Page): https://www.va.gov/get-help-from-accredited-representative/
  • "Find a VA accredited representative or VSO": https://www.va.gov/get-help-from-accredited-representative/find-rep/

Toggle to reveal remaining supporting artifacts that are pending

- [User flows](Insert URL here between the parenthesis) - [Epics/issue that may be helpful](Insert URL here between the parenthesis)

Definition of Done

### A11y Tasks
- [x] **Outreach Initiated for Pilot:** @eli-oat **reached out to the ARM team** They said they weren’t ready to move forward, yet, but they’ve checked in and I’ll follow up in a bit to schedule something." via Slack 3/4/2024
- [x] **Recommendation:** Two Pilots (1) Mobile and (1) Desktop by @coforma-jamie
- [x] Coordinate to schedule with a CAIA a11y: Eli available (@coforma-terry tagged @eli-oat in research ticket 2/29/2024 for awareness)
- [x] **Determine date and time for pilot(s):** Between 3/118 and 4/1/2024 (ARM availability)
- [x] **Pilot 1 of 2:** 03/25/2024 assigned to: Jamie placeholder scheduled with JF ; Jamie is planning: Windows+JAWS
- [x] **Pilot 2 of 2:** 03/26/2024 assigned to:Eli placeholder scheduled with JF; Eli is planning: Phone+iOS
- [x] **Requesting assistive tech devices to be used** Screen Readers & Magnification `Pending Perigean Specific AT Info` c/o Team - check [Google Sheet](https://docs.google.com/spreadsheets/d/1MJB_o5qDzjHY9NYnGriGV_SQ1AVmfi7_itcT5Iiq8ho/edit?usp=sharing)
- [x] **HOLD**: Set a11y Capacity Calendar `HOLD` for the Pilots `Will Need Invite and Zoom from ARM Team`
- [x] **INVITATION**: Pilot invite needed for CAIA a11y Capacity Calendar: [The person sending the invite would add this to the guest section of the invite.](c_f982350fc73a8162a1088ab14d00dd37335b2b559a60d6b67eb8021416f81c2a@group.calendar.google.com) If they are a CAIA team member they can simply invite the CAIA A11y Capacity Calendar.
- [x] **Review research plan** (3/6/2024 available) - [Research Plan RCVD](https://docs.google.com/document/d/13H_3v6NcwnYnnT87ggvg4n1IquCU5PduEZf0BdP0O-Q/edit?usp=sharing) Jamie `DONE`
- [x] **Review Conversation Guide** (3/7/2024 available) [Conversation Guide RCVD](https://docs.google.com/document/d/1wXs2HQeyd8Z41BBvYzXdmgv-6bY1Mt12KSpKtcSgYKA/edit?usp=sharing)
- [x] **Review Recruitment for Devices Used** `TBD`
- [x] **Receive assistive tech devices to be used** `Screen Readers` `Magnification`
- [x] **Run** pilot on 3/25/2024: CAIA A11ys to share verbal feedback during the session
- [x] **Run** pilot on 3/26/2024: CAIA A11ys to share verbal feedback during the session
- [x] **Review**  Live site as of 3/5/2024

[coforma-jamie]

A suggestion, @jfinnerty13 - perhaps we end up doing two pilots? One on mobile, with @eli-oat (who has access to various mobile devices - probably iOS/VoiceOver), one on desktop with me (probably on JAWS)? That could give you a better sense of what issues you might encounter during the study.

I can run a pilot from VoiceOver + MacOS, JAWS + Windows, NVDA + Windows.

[eli-oat]

Yeah! Building off of what @coforma-jamie said we can either do 1 pilot that hits a couple things, or multiple pilots each focused on a specific sort of assistive tech.

I'm able to run a pilot from VoiceOver on iOS, macOS, Android TalkBack or JAWS on Windows.

[jfinnerty13]

That sounds good to me :) I'm going to be starting the test plan this week - will send you both the plan and would love your input on content, device, choices, and we can go from there? @eli-oat @coforma-jamie

[coforma-jamie]

That sounds great! Link it here so we can keep everything together :)

[jfinnerty13]

@coforma-jamie @eli-oat @sara-amanda started drafting up the research plan today and have some questions please. I added you to my document and will tag you as well. Link to document

1. I broke out the study into three cohorts: low vision magnification users, alternative navigation users, and screen reader users based on the Platform website direction. Does that sound correct?
2. If that does look correct, any recommendations on screener questions for 'Alternative navigation users'? I saw options for magnification and screen readers but not this audience.
3. If there's anything else in the "Recruitment approach" of the document that you recommend I change please let me know.

Thanks!

[coforma-jamie]

@jfinnerty13 we left comments for you in your doc, please take a look!

[jfinnerty13]

Conversation guide together and ready for review - going back to comments in research plan now. Thanks! @coforma-jamie @eli-oat @sara-amanda

[sara-amanda]

Supporting Artifacts

Links added to the Pilot Session Section (copied below).

• Research plan
• Research conversation guide
• LIVE URLS
  • URLS Taken from Conversation Guide Google Doc Comment on 3/12/2024
  • "Get help from a VA accredited representative or VSO" (Landing Page): https://www.va.gov/get-help-from-accredited-representative/
  • "Find a VA accredited representative or VSO": https://www.va.gov/get-help-from-accredited-representative/find-rep/

[jfinnerty13]

For the pilot sessions @eli-oat did you want to use Apple or Windows for your mobile pilot? Then I was thinking having @coforma-jamie do the other iOS on desktop.

Between the 18th - 29th are there dates and times that work best for you or should I take a look at ya'lls calendars and find time?

Thanks!

[coforma-jamie]

It'll be easiest if you check our calendars and send times, we can work from there :)

[jfinnerty13]

Sounds good -

@coforma-jamie found time 3/25 10 - 1130 AM EST @eli-oat found time 3/26 12 - 130 PM EST

Let me know if those times don't work. Otherwise resurfacing this question for you both: @eli-oat did you want to use Apple or Windows for your mobile pilot? Then I was thinking having @coforma-jamie do the other iOS on desktop.

Thanks!

[coforma-jamie]

@eli-oat, how about you use iPhone+iOS, and I use Windows+JAWS?

[eli-oat]

Sounds good to me, @coforma-jamie 🥳

[jfinnerty13]

Updated - thank you so much both of you!

[sara-amanda]

Calendar and Ticket Updates

• [x] Ticket moved from ready to in progress
• [x] Updated ticket based on the scheduling Google Sheet from @jfinnerty13 for both pilots.
• [x] Calendar placeholders set on CAIA A11Y Capacity Calendar
• [x] Official Invite with Zoom for Pilots: Pending
• [x] AT Tech Preference for Pilots: Pending
  • We know it will be screen readers and magnification, but we don't know the specific AT or device type just yet.
    • Eli is planning: Phone+iOS
    • Jamie is planning: Windows+JAWS

@eli-oat @coforma-terry @coforma-jamie

[jfinnerty13]

@sara-amanda @coforma-jamie @eli-oat @coforma-terry

• Links above for materials are correct, and all documentation we have has been provided. We do not have user flows or epics.
• Sent Pilot sessions to Sara and Terry this morning.
• We'll be testing screen readers primarily at both sessions with magnification time permitting.

Let me know if there's anything I missed - thanks!

[coforma-jamie]

Having some technical issues on my GFE (the VPN doesn't want to connect to va.gov!), so we decided that I'll use MacOS + VoiceOver for my pilot session.