[a11y support] Chatbot: Review research for any AT feedback of note

[sara-amanda]

CAIA Research Review: Chatbot Prescription Feature

A11y/User Experience Review

Inquire:

• [ ] Chatbot mental models for chatbot interactions - does it satisfy accessibility needs?
• [ ] Do we need a new model for chatbot interaction?

Review

Blind and vision-impaired users frequently reference form-type interactions we have read, has this been the case in the research studys?

We could take a look at:

• [x] Rx Round 1 readout
• [x] Rx Round 2 readout
• [ ] Rx Round 3 readout ~when~ available @coforma-jamie 10/6/2023 add Tested with 4 AT users (SR).

Inquire

• Did they test with AT users? Did Angela assist since this ties to MHV?
  • [ ] Yes
  • [x] No

Research Plan

Research Plan for VA Chatbot Prescription Skill Feature

Review of research:

• No mention of AT.
• Observations from participants, and recommendations from researchers, that could pertain to us:

  Voice

• Users wanted an additional visual cue for the microphone, so that they know it’s working properly.
  • “I’m very reliant on the output indicator from other speech detect software.”
• Users wanted a mute option, or the ability to only have the chatbot speak when they choose to use the voice feature.
  • “I don’t know how I feel about any webpage making noise without prompting me or asking me.”
• Users wanted to actively choose to use the voice feature or not. The main frustration was that the voice automatically spoke, and there wasn’t an option to turn it off. Recommendation to give the user freedom to choose between voice and typing.
  • “My friend who is legally blind would use the mic”

    Exits

• All sighted users understood when they were leaving the prescription skill. Need to make sure AT users do too.

  Number of prescriptions

• Concern over showing only one prescription at a time. “It said two…so show me both.”
  • But other users understood the one-at-a-time pattern.
  • “I assume that it’s going to go one by one and ask me if I want to refill it….I like that it goes one by one”. Older vets liked the pattern more than younger vets. Recommendation to create a “refill all” option.

    Prescriptions list

• This also shows one at a time, and that frustrated users.
  • “I guess one at a time works for the elderly, but I expected to see a [full] list”

    Tracking prescriptions

• They list the USPS tracking number, but don’t actually link to the website. Recommendation to include that link when available.

[coforma-jamie]

I reviewed the Prescription Skill Participant Screeners, and it doesn't appear that AT users were part of the study.

The screener required that the participants:

• were familiar with VA.gov and the prescription refill process
• had access to a laptop/desktop, had Zoom downloaded, could share their screen, and were fluent in English
• demographics:
  • equally women and men
  • 2/3 non-white
  • more older than younger

There was no mention of recruiting participants that used assistive technologies to access VA.gov.

[coforma-jamie]

@sara-amanda and @SarahKay8 - my research review is in the body of this ticket. We can use them during our audit.

[sara-amanda]

Research Round 3

• [x] Research Artifact Available for Review @coforma-jamie (link to comment in 65945)
• [x] Tested with 4 AT users (screen readers) in Round 3. (link to comment about AT users)
• [x] Facilitator's guide had strong focus on accessibility.
• [ ] Pending findings (Jamie reached out 10/10/2023)

[coforma-jamie]

@sara-amanda, I reviewed the available artifacts (guide and screener) and commented back to the team in the mentioned ticket. When that testing session is over and there are findings for us to review, I assume they'll let us know?