[Higher Level Review v2] VSA-QA Test Plan

vhenry07 commented 3 years ago

[Product Managers: Open this ticket at least 3 Sprints (6 weeks) before you expect a Platform Staging Review of this Product/Feature. Then:

Replace title's [Product/Feature] with your actual product/feature name.
Add yourself to Assignees
Fill-out sections below.]

See also: Relevant VSA-QA process doc for details.

Product/Feature summary

[Provide brief summary of product/feature to be tested on Staging by VSA-QA. Also, be sure to include links to the Product/Feature Outline, Epic, Pre-Launch Checklist, and Design tickets/artifacts.]

Product/Feature Outline: [provide link]
Product/Feature Epic: [[Link to ticket]](https://github.com/department-of-veterans-affairs/va.gov-team/issues/28113)
Product/Feature Copy deck(s): [provide link, or type "In Design prototypes"]
Product/Feature Design prototypes: [Mockups]

Product-team SMEs

[if different from (or more specific than) VSA Master Org Chart]

(Leave blank to implicitly refer to VSA Master Org Chart (linked above) Design/Research: Kevin Stachura/Christian Valla Content/IA: Beth Potts Front-end: Robin Garrison Back-end: Michel McDonald QA: Tze-chiu Lei

Proposed use-cases and Staging test-accounts

[Provide specific Use Cases (all the different interactive user-flows and data-specific states) to be tested. Use this Product Use Cases template -- copy the template's markdown into a new .md file inside your Product folder [next to Product Outline file], then start updating.

IMPORTANT for authenticated products (requiring sign-in): Staging test-accounts must be provided for all authenticated scenarios, and Staging-API test-data created to match the account-specific scenarios.]

Link to Use Cases doc: [HLR v2 use cases]

How to configure this issue

[] Product Manager added to Assignees
[ ] Attached to a Milestone (when will this be completed?)
[ ] Attached to Epic (what body of work is this a part of? possibly Ongoing Maintenance)
[ ] Labeled with Team (vsa-authenticated-exp, vsa-benefits, vsa-benefits-2, vsa-caregiver, vsa-ebenefits, vsa-facilities, vsa-public-websites, etc.)

Definition of Done

[ ] Product references provided by Product Team.
[ ] Use Cases provided.
[ ] For authenticated products, Staging test-accounts and mock-data set up for Use Cases above.
[ ] Test Cases [in TestRail] finalized based on Use Cases above.
[ ] Test Runs completed on Staging [or Drupal pre-prod enviornment].
[ ] Test Reports generated. These should be provided to VSP at least 4 workdays before Staging Review.

tlei123 commented 3 years ago

hi @vhenry07 @kevinstachura ! I have some follow-up questions re. the Use Cases...

[ ] 1. What does the User* do or see for each Use Case? E.g. how does the User 'add "I am homeless' flag', 'Update Rep name format', or 'Add SOC/SSOC Date field...', etc.?
[ ] 2. What are the specific design screen-links** for each Use Case?

Please let me know. Thanks much! :)

*There seems to be a fundamental confusion between USE-cases and TEST-cases. Descriptions should be User-oriented. ** The reason there're case-specific Link-to-designs placeholders is to specify the case-relevant screens for each Use Case, not to keep repeating the root-level link.

P.S. @pjhill , if you have any suggestions to my Use Cases documentation, [especially the Use Cases are NOT Test Cases section] please let me know. Thanks much!

vhenry07 commented 3 years ago

hi @vhenry07 @kevinstachura ! I have some follow-up questions re. the Use Cases...

[ ] 1. What does the **User*** do or see for each Use Case? E.g. how does the User 'add "I am homeless' flag', 'Update Rep name format', or 'Add SOC/SSOC Date field...', etc.?

These items all reference tasks completed by front end engineering. For the homeless flag, the user will see the question "Are you experiencing homelessness .." and be able to check a box.
Rep name - User will enter Representative name and contact info
Date - User will enter date
Contact Info - User will be able to enter/update their contact info
Write-In issues - User will be able to write in a condition/issue

[ ] 2. What are the specific design screen-links** for each Use Case?

Homeless flag - Slide 9 (HLR Contact Info)

Rep name - Slid3 32 (Informal Conference)

Date field - Slide 33 (Informal Conference)

Contact Info - Slide 14 (HLR Contact Info)

Write-In Issues - Slide 19 (HLR Contested Issues) Please let me know. Thanks much! :)

*There seems to be a fundamental confusion between USE-cases and TEST-cases. Descriptions should be User-oriented. The reason there're case-specific Link-to-designs placeholders is to specify** the case-relevant screens for each Use Case, not to keep repeating the root-level link.

P.S. @pjhill , if you have any suggestions to my Use Cases documentation, [especially the Use Cases are NOT Test Cases section] please let me know. Thanks much!

tlei123 commented 2 years ago

@vhenry07 , when you open your Staging Review ticket, the QA-artifact links below should be copied there.

NOTE: I couldn't log into Staging w/ my Mac Safari. IF any of your team-members has 20-30 min. in the next few days to test that browser, please share your TestRail creds with him/her and have him/her tag me in this ticket -- I'll do a quick screenshare to get him/her started.

QA test-runs now completed. 0% Passed - 4 defects logged: #32331, #32332, #32333, #32344.

QA-artifacts:

Test Plan: https://dsvavsp.testrail.io/index.php?/plans/view/2485
Test Case: https://dsvavsp.testrail.io/index.php?/suites/view/6&group_by=cases:section_id&group_order=asc&display_deleted_cases=0&group_id=2910
Coverage for References (Cases) report: https://dsvavsp.testrail.io/index.php?/reports/view/276
Summary (Defects) report: https://dsvavsp.testrail.io/index.php?/reports/view/277

cc @pjhill @ddzz

tlei123 commented 2 years ago

Re-opening for new UI-flows & 2nd round of tests.

tlei123 commented 2 years ago

@vhenry07 , 2nd round of test-runs now completed. Updated report link is below, and also updated in Staging Review ticket #34244:

Updated Summary (Defects) report

department-of-veterans-affairs / va.gov-team