search

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
281 stars 202 forks source link

[CAIA A11y Research]: Benefits Decision Review, NOD (February 2024) #74234

Closed eileen-coforma closed 7 months ago

eileen-coforma commented 8 months ago

Project Overview

[!NOTE]

High-Level Overview: Benefits Decision Review, NOD

Findings

Comments/findings provided via the provided Mural Board

Study Details

  • Pilot: 2/7/2024 Details in Ticket #74263
  • Research Sessions: STUDY EXTENDED 2/8-2/15/2024 (from 2/14/2024)
  • Time of Day: between 9 a.m. and 2 p.m. ET
  • Status: COMPLETED - invites for newly added sessions in Google Doc
  • Assistive Technology Users: Original: 10 **NEW: 14
    • This increased due to new sessions.
  • Assistive Technology Types: ACTUAL: 6 (includes pilot)
    • 4 VoiceOver 1 JAWS 1 TTS
    • (CAIA Sessions ONLY and includes pilot)
    • Task Lists: See task lists for the Benefits Decision Review Team and CAIA A11ys for next steps/definition of done

CAIA Assignee(s)

A11y Availability

Toggle to availability info. _Thank you for reaching out to CAIA! We are looking forward to supporting your study. Thank you for being flexible with your study, as we had multiple studies during the original timeframe.._ _We will do our very best to work with you and @artsymartha68, to support your research sessions, based on a11y availability. Should you have any questions, issues, or concerns please do not hesitate to contact @coforma-terry (CAIA A11Ys) and/or @artsymartha68 (OCTO) as soon as possible._ _We have blocked out your time slots on our [CAIA A11y Capacity Calendar](https://calendar.google.com/calendar/embed?src=c_f982350fc73a8162a1088ab14d00dd37335b2b559a60d6b67eb8021416f81c2a%40group.calendar.google.com&ctz=America%2FNew_York) for the time span you have provided._

Team Info

Toggle to reveal additional team information ### Do you have an accessibility champion? Yes, we have a champion ### If you have an accessibility champion, what's their name? David Kennedy ### Is anyone on your team interested in becoming an accessibility champion? _No response_ ### Calendar checking - [X] Based on this calendar, confirm we have specialists who can support your study in your desired timeframe - [X] Confirm you are filing this support ticket at least 1 week in advance of said timeframe, so we have enough heads up to provide support ### Have you reviewed the CAIA capacity calendar? - [X] Yes, we have reviewed the CAIA capacity calendar and can confirm specialists are available to support in our research timeframe

Research Sessions

Toggle to reveal research session information ### Timeframe - **Pilot Session**: 2/7/2024 `CONFIRMED` - **Timeframe**: 2/8/2024 - ~2/14/2024~ `EXTENDED` 2/15/2024 ### Length of sessions - **Session length:** `60` minutes - **Buffer time between sessions:** `60` minutes - **Maximum sessions per day:** `2` ### Sessions ### Testing Schedule, Device and AT Combinations The full testing schedule, assignments, and AT/device information can be found in this [Google spreadsheet](https://docs.google.com/spreadsheets/d/1JO8fJZlUAF_qSN2DCrM2aOOqIu88Q09SZ8jWIsp1TpU/edit#gid=0). This is the testing source of truth for this study. The final schedule is pasted here for long-term record-keeping: | Perigean Tracking number | Session Date (ET) | Status | Moderator | A11y Specialist | Quote taker | Observer 1 | Observer 2 | Device | Screenreader type | Years visually impaired | Comfortability with AT | Notes | | ------------------------ | ----------------- | ------------------------ | --------- | ----------------- | ----------- | ----------- | ----------------- | ------------------ | ------------------ | ----------------------- | ---------------------- | -------------------------------------------- | | 0 | 2/7 (Wed) 11AM | Pilot | Eileen | DK | Sade | Ruben | Eli Mellen (CAIA) | Computer / MacOS | VoiceOver | N/A | N/A | Either Evan or Eli will be the PPT for pilot | | 5 | 2/12 (Mon) 10AM | Completed | Eileen | DK | Sade | Sara (CAIA) | Julie S | Computer / Windows | JAWS | 10+ | Very comfortable | | | 7 | 2/13 (Tue) 9AM | Completed | Eileen | DK | Sade | Ruben | Jamie (CAIA) | Computer / MacOS | VoiceOver | 10+ | Very comfortable | | | 8 | 2/14 (Wed) 2PM | Completed | Eileen | DK | Ruben | Heather | Jamie (CAIA) | Computer / MacOS | VoiceOver | 5-10 | Very comfortable | | | 9 | 2/15 (Thu) 10AM | Cancelled by participant | Eileen | DK | Ruben | Sade | Jamie (CAIA) | Computer | Talkback | 2-5 | Very comfortable | | | 10 | 2/15 (Thu) 12PM | Completed | Eileen | DK | Ruben | Sade | Eli Mellen (CAIA) | Computer / MacOS | VoiceOver Edge TTS | 2-5 | Very comfortable | Low vis | | 12 | 2/16 (Fri) 12PM | Cancelled by participant | Eileen | Eli Mellen (CAIA) | Ruben | | | Computer | JAWS | 2 | Very comfortable | | | 16 | 2/23 (Fri) 4PM | Completed | Eileen | Evan (CAIA) | Sarah S. | Sade | Julie P | Computer | VoiceOver | 10+ | Very comfortable |

Supporting artifacts (if any)

Toggle to reveal supporting artifacts information - [Product outline](https://staging.va.gov/decision-reviews/submitted-appeal/) - [User flows](Insert URL here between the parenthesis) - [Prototype or mockups](https://staging.va.gov/decision-reviews/submitted-appeal/) - [Research plan](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/decision-reviews/Notice-of-Disagreement/Research/012024-accessible-submission-download/research-plan.md) - [Research conversation guide](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/decision-reviews/Notice-of-Disagreement/Research/012024-accessible-submission-download/conversation-guide.md) - [Epics/issue that may be helpful](https://github.com/department-of-veterans-affairs/va.gov-team/issues/71952)

...

Next steps

### Benefits Decision Review Team - Next Steps
- [x] Post a link to this ticket in the accessibility-help Slack channel and tag Terry Nichols.
- [x] Your team's researchers and observers read through [platform guidance on research with assistive technology users](https://depo-platform-documentation.scrollhelp.site/research-design/research-with-assistive-technology-users)
- [x] **PILOT**: Requested for 2/7/2024
- [x] **LINK TO PROTOTYPE**: Prototype will be reviewed in the following environment:`STAGING` with CAIA A11y(s) during the pilot session. [Staging Prototype Link](https://staging.va.gov/decision-reviews/submitted-appeal/)
- [x] **INVITE CAIA A11YS**: CAIA Specialists will need to be assigned to your research, be sure to add the respective a11y specialist(s) to your session invites (include the Zoom or Teams link). Be sure to invite the CAIA A11y Capacity Calendar - if possible. Original Invites Received 2/6/2024 at 10 a.m. `REPLACED GENERIC HOLDS WITH INVITES`
- [x] **SEND NEW INVITES FOR ADDED SESSIONS TO A11YS** based on new sessions added to the Google Sheet on 2/12/2024 for 2/15/2024.
- [x] **AT PARTICIPANT COUNT AND DETAILS:** See table in the body of this ticket (_above in the Device & AT Combinations section_). Provide specialists your participant details like skill level, operating system, assistive tech used, browser `Prior to Sessions - If Possible` `Partial Info Prior to Sessions Helps`
### CAIA A11Y - Next Steps
- [x] Ticket formatting and updates with task lists 1/23/2024.
- [x] **REVIEW PROTOTYPE:** Prototype is now available to be tested in the following environment:`STAGING`
- [x] **REVIEW ARTIFACTS:** CAIA Specialists review your conversation guide and research plan.
- [x] **CALENDAR HOLDS - RESEARCH WINDOW** Put initial calendar holds on [CAIA A11y Capacity Calendar](https://calendar.google.com/calendar/embed?src=c_f982350fc73a8162a1088ab14d00dd37335b2b559a60d6b67eb8021416f81c2a%40group.calendar.google.com&ctz=America%2FNew_York) `GENERIC HOLDS CURRENTLY`
- [x] **PILOT INVITE** Pilot session with team as requested for 2/7/2024 `Invite sent 1/23 for 2/7 to a11ys`.
- [x] **PILOT SESSION** Participate in a pilot session with team as requested for 2/7/2024.  **See Ticket** `74263`
- [ ] https://github.com/department-of-veterans-affairs/va.gov-team/issues/74263
- [x] **SIGN-UP GOOGLE SHEET** [A11ys can sign up for sessions on the sign-up sheet.](https://docs.google.com/spreadsheets/d/1JO8fJZlUAF_qSN2DCrM2aOOqIu88Q09SZ8jWIsp1TpU/edit#gid=0) as of 2/6/2024 and invites will follow.
- [x] **CALENDAR HOLDS** Put official calendar holds on [CAIA A11y Capacity Calendar](https://calendar.google.com/calendar/embed?src=c_f982350fc73a8162a1088ab14d00dd37335b2b559a60d6b67eb8021416f81c2a%40group.calendar.google.com&ctz=America%2FNew_York) `PENDING INVITES FROM TEAM/PERIGEAN`
- [x] **REPLACE CALENDAR HOLDS** with official Perigean invites, based on Google Sheet signups.
- [x] **PARTICIPATION:** Participate in User Research Sessions: `START`2/8/2024 `END` 2/15/2024
- [x] **NEW INVITES FOR ADDITIONAL SESSIONS** `Pending` based on Google Sheet 2/12/2024.
- [x] **A11Y FEEDBACK:** CAIA Specialists created a [research notes template](https://docs.google.com/document/d/1v2PRHzq5NtFBZpP7H9_6JIh1ESfpBxZyc1RrNMn-nc0/edit?usp=sharing) for these sessions.
- [x] **POST-RESEARCH:** CAIA Post research, specialists will add their findings to the team's Mural (along with other observers)
- [x] **POST-RESEARCH**: CAIA post-research, specialists will copy the final [schedule table](https://docs.google.com/spreadsheets/d/1JO8fJZlUAF_qSN2DCrM2aOOqIu88Q09SZ8jWIsp1TpU/edit#gid=0) to this ticket for record-keeping
- [x] **HISTORICAL REF.** [Copy Google Sheet Sessions to this Ticket](https://docs.google.com/spreadsheets/d/1JO8fJZlUAF_qSN2DCrM2aOOqIu88Q09SZ8jWIsp1TpU/edit#gid=0) for Archival Purposes Post-Research
sara-amanda commented 8 months ago

Research Ticket Updates

cc: @coforma-terry

eli-oat commented 8 months ago

@eileen-coforma is there info about what assistive tech would be useful for me to use during the pilot session?

I'm able to use:

eileen-coforma commented 8 months ago

@eli-oat Most of our participants will be using VoiceOver, and a few JAWS. I think VoiceOver would be most helpful for the pilot session.

eli-oat commented 8 months ago

Wicked! I can do that. Thanks so much!

sara-amanda commented 8 months ago

Research Ticket Updates

cc: @coforma-terry

coforma-jamie commented 8 months ago

We'll provide feedback via Mural stickies, along with the other observers, since DK is an on-team a11y specialist. We'll also attend team debriefs.

eileen-coforma commented 8 months ago

@sara-amanda @coforma-jamie The study has been extended to 2/20.

There's one session open at 2/20 2PM ET. Please let me know if you'd be able to, or like to attend.

CC: @coforma-terry

coforma-jamie commented 8 months ago

I'm unavailable, maybe @sara-amanda can attend?

eileen-coforma commented 8 months ago

Hey CAIA team, We've extended our study in order to test an iteration on the design. We've extended our schedule all the way until 3/4, with the expectation to complete 15 sessions (we are at 9 so far).

We're expecting DK to be our support, and will ask specifically when he's not available. Otherwise, you're welcome to join as observer.

CC: @coforma-terry

eileen-coforma commented 7 months ago

@sara-amanda Is it possible to get support for 2/23 at 4PM ET? Our sessions got extended to 4/4, and this is one where we might have DK available.

CC: @coforma-terry

sara-amanda commented 7 months ago

Is it possible to get support for 2/23 at 4PM ET? Our sessions got extended to 4/4, and this is one where we might have DK available

@eileen-coforma I am sure we can have someone assist. If you wanted to send the invite our way, I have a hold on the calendar now. Thanks! cc: @coforma-terry

eileen-coforma commented 7 months ago

Terry mentioned that Evan would be able to take the session. Thank you!

coforma-jamie commented 7 months ago

All CAIA-involved sessions are complete, so I'll close this ticket. Glad to support the team @eileen-coforma!