search

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
283 stars 204 forks source link

[CAIA A11y Research]: Authentication Experience, My VA (dot indicator usage) (Early-Mid March 2024) #73932

Closed mattmarino-adhoc closed 7 months ago

mattmarino-adhoc commented 9 months ago

High-Level Overview

[!NOTE]

  • Initial Kickoff: Feb. 1, 2024 at 11 - 11:30 a.m. ET
  • Research Timeframe: 3/11-3/18/2024
  • Time of Day: 11:30 a.m. to 3:30 p.m. ET (range)
  • Pilot: 2/23/2024 COMPLETED 75143
  • Assistive Tech Users: 5 AT Users
    • As of 3/25/2024, only the pilot and one AT user session have been recorded and qualified as AT testing.
      • Per Matt, there is a new session scheduled for 3/28 at 10 a.m. ET with a screen reader user. (invite added to A11Y Capacity Calendar on 3/25 for 3/28)
      • Matt is hoping to get three more users, per Slack on 3/25.
      • Matt sent over one of the three additional sessions on 3/27.
  • Artifacts Available: 1/18/2024
  • Testing Environment: ~Figma~ CodePen coming week of 2/6/2024
  • Assistive Tech Being Used: Screen Readers Types - TBD during sessions via ask by MM Desktops/Laptops
  • Task Lists: See task lists for next steps/definition of done and CAIA A11ys
  • Cancelations: Double-check for last-minute cancelations in the team's Slack Canvas

CAIA Assignee(s)

[!NOTE] There was a sudden influx of studies, taking place prior to yours, delaying the processing of this ticket request. Thank you so much for your patience.

Toggle to view information about the CAIA Team. - @sara-amanda - @coforma-terry - @coforma-jamie (pilot) - @eli-oat - @EvanAtCoforma

About the Employee Experience Team

Toggle to view information about the VFS Team. - Team name:Authentication Experience (My VA) - OCTO-DE product owner: Samara Strauss - Product manager: Anastasia(AJ) Jakabcin - Designer: Matt Marino - FE engineer: Allison Liu - Product/team Slack channel: #accountexp-authexp
Toggle to view more intake research ticket information ### Do you have an accessibility champion? No, but we are interested in training one! ### If you have an accessibility champion, what's their name? _No response_ ### Is anyone on your team interested in becoming an accessibility champion? Me (Matt Marino) ### Calendar checking - [X] Based on this calendar, confirm we have specialists who can support your study in your desired timeframe - [X] Confirm you are filing this support ticket at least 1 week in advance of said timeframe, so we have enough heads up to provide support ### Have you reviewed the CAIA capacity calendar? - [X] Yes, we have reviewed the CAIA capacity calendar and can confirm specialists are available to support in our research timeframe ### When is the timeframe for your testing? Before the end of February

Research Sessions

[!TIP]

  • Providing CAIA A11ys with the time slots in advance for AT users, will allow a11ys to avoid conflicts on the CAIA A11y Capacity Calendar, and reach out to other a11ys, should CAIA not be available.
  • Providing CAIA A11ys with the technology and devices the users will be testing with, will allow CAIA to prepare in advance for the sessions.
Toggle to view research timing information. ### Timeframe - **Pilot Session**: ~2/9/2024~ `Invite Updated on CAIA A11y Calendar` `Being Rescheduled` for 2/16/2024 or the following week. NEW DATE/TIME: 2/20/2024 1-2 p.m. ET; changed as of 2/20 to 2/23/2024 at 3:30 p.m. with Jamie. - **Timeframe**: 3/`##`/2024 - 3/`##`/2024 ### Length of sessions - **Session length:** `pending` minutes - **Buffer time between sessions:** `pending` minutes - **Maximum sessions per day:** `pending`

Testing Schedule

[!TIP]

  • We can check against an additional resource in Slack.
  • Double-check for last-minute cancelations in the team's Slack Canvas
  • FYI: Matt is going to be asking about the specific AT being used at the start of each session, since that information is not yet available in detail.

CAIA A11Ys will be looking for device and AT information. We realize that some of this information will not be available until the day of the session.

Please toggle and use the following table to enter the following information for each participant: Date and Time, Assistive tech (AT), Device and Operating System, Browser, and Skill level:

View a11y Testing Coverage

Toggle to view the AT testing schedule P# | Date and Time (ET) | AT | Device and OS | Browser | Skill | CAIA A11ys Available | Notes from [Slack Canvas](https://dsva.slack.com/docs/T03FECE8V/F06NLT96GNR) on AT | Notes from Session ------------------|------------------|--------------|--------------|--------------|--------------|--------------|--------|------| Pilot | 02/23/24, 3:30-4:30pm | VoiceOver | Desktop, MacOS | Safari | Intermediate | @coforma-jamie | Pilot: 2/23/2024 https://github.com/department-of-veterans-affairs/va.gov-team/issues/75143 | ✅ **COMPLETED** P2 | ~~3/11/24, 11:30am – 12:30pm~~ **No show** | VoiceOver, JAWS, etc `TBD` | Desktop/Laptop | Safari, Chrome, etc. `TBD` | Novice, Intermediate, Advanced `TBD` | @coforma-jamie | (Screen reader/magnification, zoom, or font resizing) | 🔴 `P2` **No Show** P5 | ~3/12/24, 10:00am – 11:00am~ **DNQ** | VoiceOver, JAWS, etc `TBD` |Desktop/Laptop | Safari, Chrome, etc. `TBD` | Novice, Intermediate, Advanced `TBD` | @eli-oat | (Screen reader/ Uses magnification, zoom, or font resizing to browse the internet) | 🔴 `P5` DNQ [See P5 in Mural - Not an AT User](https://app.mural.co/t/departmentofveteransaffairs9999/m/departmentofveteransaffairs9999/1709061562188/1dfb9cdf6dbb59004be7cfff6714b3c704d7fa37?sender=u32ca7987ad64375cb5bb9275) P9 | 3/13/24, 1:00pm – 2:00pm | VoiceOver | iPad | Safari | Beginner | @EvanAtCoforma | (Screenreader/Colorblind) | ✅ `P9` COMPLETED [Raw Notes for P9](https://docs.google.com/document/d/1w4Hlf8qSnl3LD8X199-6IFetXujPF1nto9godcK1x-s/edit#heading=h.471l8v8lfvwu) P10 | ~3/13/24, 2:30pm – 3:30pm~ **No Show** | N/A | N/A | N/A| N/A | @SarahKay8 | (Screenreader/Uses magnification, zoom, or font resizing to browse the internet) | 🔴 `P10` **No Show** P14 | ~3/18/24, 11:30am – 12:30pm~ **DNQ** | N/A | Mobile | Safari | NA | @SarahKay8 | (Screenreader/Colorblind) | 🔴 `P14` DNQ **Q&A; No testing was done.** 🆕 **P21** | 3/28/2024 10 - 11 a.m. | VoiceOver | Desktop/Laptop | Safari, Chrome, etc. `TBD` | Novice, Intermediate, Advanced `TBD` |@coforma-jamie | VoiceOver | ✅ `P21` COMPLETED 🆕 **P22** | 4/1/2024 at 10:30 a.m. | Zoomtext/Seekit | Desktop (Windows) | Safari | Intermediate | @SarahKay8 and @EvanAtCoforma | (Screen Reader User)TBD | ✅ `P22` COMPLETED - **Consideration**: You can also consider asking participants more specific AT and device combinations. - **For instance:** TalkBack on Samsung Galaxy 8, or VoiceOver on iPad with magnification.

CAIA A11ys will review the coverage needed, and self-assign, based on a11y availability.

Supporting artifacts

Toggle to view the supporting artifacts - [Product outline](https://github.com/department-of-veterans-affairs/va.gov-team/tree/master/products/identity-personalization/my-va#product-outline-my-va) - [User flows](https://www.figma.com/file/15yOY4VEzitxm5tRMDiAzz/My-VA?type=design&node-id=0-7253&mode=design) - [Prototype](https://www.figma.com/proto/15yOY4VEzitxm5tRMDiAzz/My-VA?node-id=205-27290&starting-point-node-id=205%3A27290) - [Figma File](https://www.figma.com/file/15yOY4VEzitxm5tRMDiAzz/My-VA?type=design&node-id=196%3A26639&mode=design&t=3uRwcSw5HvV7vL1R-1) - Codepen prototypes: - [Two dots](https://bit.ly/49Bvi7O) - [Badge indicator](https://bit.ly/3IeFwyU) - [Slim Alert](https://bit.ly/48lBwHG) - [Research plan](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/identity-personalization/onsite-notifications/dot-indicator/research/2024-03%20Dot%20Indicator%20and%20Slim%20Alert%20Usability%20Study/dot-indicator-research-plan.md) - [Research conversation guide](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/identity-personalization/onsite-notifications/dot-indicator/research/2024-03%20Dot%20Indicator%20and%20Slim%20Alert%20Usability%20Study/dot-indicator-convo-guide.md) - [Screen reader conversation guide](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/identity-personalization/onsite-notifications/dot-indicator/research/2024-03%20Dot%20Indicator%20and%20Slim%20Alert%20Usability%20Study/dot-indicator-convo-guide-for-screen-readers.md) - [Epics/issue that may be helpful](https://github.com/department-of-veterans-affairs/va.gov-team/issues/63788)

Next steps

### A11Y Tasks
- [x] **INTAKE TICKET:** @sara-amanda to process intake ticket
- [x] **INITIAL MEETING:** Meeting with @coforma-terry and @mattmarino-adhoc 2/1/2024 re: conversation guide/research plan for assistive tech user
- [x] **FEEDBACK REQUESTED:** Review [Conversation Guide](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/identity-personalization/onsite-notifications/dot-indicator/research/dot-indicator-convo-guide-for-screen-readers.md) prior to pilot for screen readers and provide Matt with feedback, per request 2/1. [Feedback Completed 2/1 by Jamie](https://github.com/department-of-veterans-affairs/va.gov-team/issues/75143#issuecomment-1922221752)
- [x] **PILOT SESSION: SCHEDULE** Pilot session 2/9 with Jamie or Eli (both accepted and are available) - WILL BE RESCHEDULED - WILL NOT TAKE PLACE ON 2/9
- [x] **Reschedule Pilot** Per Matt, we can schedule this for 2/16/2024 or the following week. Inquiring on 2/13 with Eli and Evan on availability 2/20/2024 1-2 p.m. ET; changed to 2/23/2024 at 3:30 p.m. with Jamie (as of 2/20)
- [x] **Update Calendar** to reflect new pilot date/time `pending new invite`
- [ ] https://github.com/department-of-veterans-affairs/va.gov-team/issues/75143
- [x] **CALENDAR HOLDS** Update A11y [CAIA Capacity Calendar](https://calendar.google.com/calendar/u/0/embed?src=c_f982350fc73a8162a1088ab14d00dd37335b2b559a60d6b67eb8021416f81c2a@group.calendar.google.com&ctz=America/Chicago) with holds for user research sessions with AT, when dates become available.
- [x] **REMOVE CALENDAR HOLDS** Remove holds on A11y [**CAIA Capacity Calendar**](https://calendar.google.com/calendar/u/0/embed?src=c_f982350fc73a8162a1088ab14d00dd37335b2b559a60d6b67eb8021416f81c2a@group.calendar.google.com&ctz=America/Chicago) and replace with invitations from project team (or Perigean)
- [x] **RESEARCH NOTES** Create [CAIA notes document](https://docs.google.com/document/d/1w4Hlf8qSnl3LD8X199-6IFetXujPF1nto9godcK1x-s/edit?usp=sharing)
- [x] **Divvy user research session support with CAIA A11ys** - based on availability and technology available to support (Table to be added to this ticket, above, with AT session dates and times, so that available a11ys can sign up.)
- [x] **Review research folder and assets** [Research plan](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/identity-personalization/onsite-notifications/dot-indicator/research/dot-indicator-research-plan.md) and [Conversation SR Guide](https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/identity-personalization/onsite-notifications/dot-indicator/research/dot-indicator-convo-guide-for-screen-readers.md)
- [x] **Additional AT Session**: New Session 3/28/2024 at 10 a.m. ET (a11y Jamie) on Calendar - Per Slack, Matt hopes to get three more.
- [x] **Participate in research** **Date Span:**`3/11/2024` - `3/18/2024` (extended new session 3/28, with hopes for three more sessions; 4/1 received on 3/27; 2 more sessions potentially)
- [x] **IF THERE ARE MORE NEW SESSIONS** Add additional 3 sessions to the calendar and table (if recruited) - as of 3/25; 1 of 3 added on 3/27/2024 for 4/1/2024 (a11y needed)
- [x] **SESSION NOTES**: Update the [CAIA notes document](https://docs.google.com/document/d/1w4Hlf8qSnl3LD8X199-6IFetXujPF1nto9godcK1x-s/edit?usp=sharing) during the sessions
- [x] **SYNC:** Connect with CAIA A11ys on research findings (talked on Slack)
- [ ] **Create/update the CAIA A11y Research Deliverable** This is dependent on the number of study participants CAIA supports. If minimal, A11Ys may share feedback only in the feedback Mural or this ticket. `not creating - all notes shared via Mural, Slack, notes doc`
- [ ] **Share the completed deliverable** with the project team via Slack: **Primary Team Slack:** [#accountexp-authexp](https://dsva.slack.com/archives/C909ZG2BB) and Ticket
### Auth Experience Tasks
- [x] **Schedule initial meeting** with Sara and Terry
- [x] **Schedule a pilot session** with CAIA a11ys 2/9 with Jamie or Eli
- [x] **Reschedule Pilot** ` **New Date/Time:** ~2/20/2024 1-2 p.m. ET~ 2/23/2024 3:30 - 4:30 p.m. ET with Jamie
- [x] [Receive feedback on Conversation Guide for Screen Readers](https://github.com/department-of-veterans-affairs/va.gov-team/issues/75143#issuecomment-1922221752) pursuant to request on 2/1
- [x] **CODEPEN PROTOTYPE:** Provide prototype in CodePen to CAIA A11ys via this ticket in the supporting artifacts section
- [x] **DATES NEEDED** See `Table 1` in the ticket above. Please provide date range and slots for AT sessions being planned.
- [x] **ASSISTIVE TECH. INFO NEEDED:** Please provide the AT the users will be utilizing during the research sessions in `Table 2` in the ticket above.
- [x] **INVITES:** Forward invites from Perigean to CAIA A11ys (Sara or Terry)
- [x] New Sesssion Invite 3/28/2024 - sent to CAIA on 3/25/2024 - Jamie will be covering this for you
coforma-terry commented 9 months ago

Assigning @sara-amanda for visibility here - thanks!

sara-amanda commented 9 months ago

Intake Received

Hi, @mattmarino-adhoc ! Just a quick note here to let you know that we received your intake ticket and will be processing it later this week. It normally isn't like this, we have had a sudden influx of studies, taking place prior to yours.

Next Steps

I'd like to set up a time to talk this sprint about the conversation guide/research plan for assistive tech users.

All the best,

Sara

cc: @coforma-terry

sara-amanda commented 9 months ago

Ticket Updated

Hi, @mattmarino-adhoc ! It was great talking to you today! Here is an update on your ticket for research.

Participant Info

This ticket has been prepped for your future testing. I have created a high-level summary at the top of the ticket, added two tables (one for your participants, so that we can schedule available a11ys) and the other to collect assistive tech. information, so that we can prep for each session.

Conversation Guide

Additionally, @coforma-jamie has completed her review of the conversation guide for screen readers. Her feedback is linked in your task list above and here.

Pilot

A ticket has been generated for the CAIA team to track the pilot work for 2/9/2024 via ticket 75143. Both @eli-oat and @coforma-jamie appear to be available for that pilot session.

Tasks

The task lists for each of us have been created, so that you can see what CAIA is doing, and if you have any outstanding tasks yourself.

We are looking forward to supporting your study!

All the best,

Sara

cc: @coforma-terry

sara-amanda commented 9 months ago

Pilot Notes

Pilot Postponed

Added Note 2/9/2024

Access information, relating to rescheduling the pilot in the pilot ticket.

Removing Note 2/12/2024

~Temporary Note: The pilot is no longer taking place on 2/9/2024. It will be rescheduled when Matt returns to the office. For all Pilot-related updates, please refer to the ticket 75143~

Pilot Ready to be Rescheduled

Pilot Rescheduled

Removing Note 2/27/2024

[!IMPORTANT] ~PILOT RE-SCHEDULED~

  • ~The new pilot has been rescheduled for: 2/23/2024 3:30-4:30 p.m. ET~
  • ~Will also note this in the Pilot Ticket 75143~

Pilot Status Update

2/27/2024

@coforma-terry

coforma-jamie commented 8 months ago

From @mattmarino-adhoc:

The dot indicator/slim alert research recruitment effort is now ready for Perigean, a kickoff call should hopefully be happening early next week

Will be on the lookout for dates later next week!

sara-amanda commented 8 months ago

Updated ticket Table 1 to reflect shift to March testing.

coforma-jamie commented 8 months ago

AT sessions will be held March 11-15, 17-18. @mattmarino-adhoc will update the participant table in this ticket, and invite @sara-amanda to the sessions (we'll assign CAIA a11ys from there).

sara-amanda commented 8 months ago

Table Updates

@mattmarino-adhoc I have updated the Testing Schedule Table with a11ys.

sara-amanda commented 7 months ago

New Sessions Being Added

Canvas Board (with participants/session updates)

Slack Updates

I have an update with my research study - we've had difficulty getting screen reader users, or at least ones that would be appropriate for testing our prototypes. I just added you to an invite for Thursday, we're trying to get 3 more screen reader users if we can. Let me know if this is manageable. Right now this is the only new test we have scheduled - 3/25 per @mattmarino-adhoc

Hey Sara, I know you're busy right now and OOO soon, but one more thing came up: we now have another session on Monday with a screen reader user, if you can help us with possibly getting someone else from CAIA? Let me know if I should reach out to someone else instead :slightly_smiling_face: - 3/27 per @mattmarino-adhoc

Updates

cc: @coforma-terry *Updated 3/27/2024

coforma-jamie commented 7 months ago

This study is complete as of April 1, 2024. @mattmarino-adhoc received CAIA feedback via Mural, Slack, and our notes document, and didn't need us to synthesize our findings further.

Thank you for working with us!