[a11y audit] Chatbot: Documentation and Testing Prep

sara-amanda commented 1 year ago

CAIA Chatbot Prep Work

[x] Meeting: VA Chatbot RX Feature Background and Sync 9/19/2023
[x] Create: Team Folder in a11y shared drive 9/20/2023
[x] Collect: Information gathering/documentation 9/22/2023 start
[x] Coordinate: VPAT Review for accessibility-related items (DSVA Slack 8/7/2023) 9/28/2023 with Jamie's assistance.
[x] Research Review Ticket: Created for a11y-related item identification with Jamie's assistance.
[x] Initial findings from SK and Jamie, shared with Chatbot team and Terry. Launch blocker found.

Next Steps

As of 10/11/2023 at 1:45 p.m. we are at 51% remaining to be audited.

[x] Share findings with project team and with governance team by EOD 10/11/2023
[x] Next staging review on 10/12/2023: CAIA is working with the audit doc to prep items prior to that date.

Aggregation of Resources R&D

[x] Full Audit Prep: Cross-Reference/Create New Chatbot Specific Audit Documentation: Playbook with audit doc, WCAG, COGA (starting 9/28/2023 - 10/9/2023)
- [x] ChatBot Audit Area Recommendation Audit Areas
  - Links to full details for how to go about reviewing the Chatbot for htis item. Colored based on audit priority column.
- [x] Relationship to Traditional Audit Testing Plan
  - These flags are based on Keyword Matches.
  - Since the ChatBot is a different flow than webpages or a form, we are testing by using WCAG and COGA to fill in the gaps.
  - Audit areas are not pages; however, if you feel you need the original testing plan structure to complete this audit, feel free to use the traditional testing plan and issues sheet linked here.
- [x] Audit Priority
  - Combines the number of impact areas (visual, auditory, cognitive, speech and physical) and the type (wcag, coga or none) for an impact weight.
- [x] Impact Areas
  - Sorted by weight
  - Visual, auditory, cognitive, speech and physical
- [x] Type
  - Relationship to WCAG and/or COGA Design Patterns
  - WCAG only
  - COGA only
  - WCAG and COGA
  - No WCAG or COGA
- [x] Topic
  - Detail provided if available from playbook chart.
- [x] Assessment Item To Test
  - Based on the self-assessment checklist.
- [x] Assessment Answer
  - Based on the assessment item in the Assessment Item to Test Column.
  - Yes (all)
  - Yes (most)
  - No (all)
  - No (some)
  - Needs further investigation
- [x] Built-In Accessibility (from playbook)
- [x] Customizable Accessibility (from playbook)
- [x] Impact Area Weight
- [x] Weighted the impact area.
- [x] Type Weight Impact Areas and Type Weight (Combo)
  - Combined the weight of the impact areas and the type for a total to sort by.
  - This was for prioritization.
- [x] WCAG
- [x] COGA Design Patterns
- [x] VPAT Status c/o Jamie
- Data Entry Fields for Team
  - [x] Tester
  - [x] Impact Area(s)
  - [x] RX/Chatbot
  - [x] Pass/Fail
  - [x] Severity/Priority from Audit
  - [x] Issue Location(s)
  - [x] "Images or Link to Images in Folder"
  - [x] Steps to Reproduce
  - [x] Notes
[x] Integrate Traditional Testing Plan into audit doc
- Same as current version.
- Edit: Description of Chatbot in flow, due to screens not being linkable sections, such as forms or pages.
- Referenced the audit buckets on the full audit sheet, flagging 12 keyword matches for prioritization.
[x] Integrate Enhanced and Traditional Issues into audit doc
- Segmented issues relating to:
  - [x] Testing Issue Area Related To: Traditional Testing Plan Col.
  - [x] Testing Plan Related ChatBot Auditable Items (combo of area, recommended playbook item and wcag/cogo) for a segmented drop-down.
  - [x] Platform Standards Specific to Chatbot: Segmented the platform standards down to a shorter list, adding in the severity level and wcag numbering for a more comprehensive dropdown, to help expedite, due to short timeframe.
[x] Pivot: Pivot tables created to keep track and report out on progress.

Documentation Prep

Documentation Location: a11y > Teams > Chatbot
- Internal: Info gathering/notes doc for Chatbot (w/ TOC)
Initial: Chatbot threads
- Primary chatbot thread
- Follow-up chatbot thread

Additional Slack Threads

For quick ref.

Tickets

Primary a11y Support Ticket for Chatbot #65945
- This ticket 65945 is the universal a11y support ticket for the VA Chatbot: Prescription Skill Feature.
  - It contains information initially collected for this task.
  - As of 10/6/2023, we are starting to use Tasks/Issues within this ticket 65945.
    - These are best viewed in Google Chrome.
    - Tasks/Issues are best viewed in GitHub and GitHub Projects.
    - ZenHub displays tasks/issues as markdown and/or URLs (if issues).
    - Tasks are checkboxes, with no assignee or labels.
    - Tasks converted to issues are able to be assigned, with labels, and additional project planning details.
    - Also, as of 10/6/2023, wea re going to be using Dev to test, as advised, based on the collaboration cycle conversation between Terry and Shira.
    - As of 10/9/2023, in order to test an authenticated version there are special instructions noted in the ticket body.
[x] ReadMe tab created within the Audit Google Sheet to address in detail items, as this process evolves.
- [x] Tickets will be spun up for failures and tasks will remain in use for tested items that passed.

Joy-Elizabeth commented 1 year ago

@sara-amanda Is there anything outstanding in our documentation and testing prep. I believe this was success in our collab cycle staging review as we had a passing score of 80%. Please confirm if there is any outstanding issues with this.

sara-amanda commented 1 year ago

@sara-amanda Is there anything outstanding in our documentation and testing prep. I believe this was success in our collab cycle staging review as we had a passing score of 80%. Please confirm if there is any outstanding issues with this.

@Joy-Elizabeth I defer to the collaboration cycle and Governance Team, here. CAIA can only point out issues found in our deliverable. Platform would be the ones to tell you if you can move forward, based on all the findings.

cc: @coforma-terry

sara-amanda commented 1 year ago

Closing this documentation ticket.

department-of-veterans-affairs / va.gov-team