[QA] Test Plan - SSO Cerner

tlei123 commented 4 years ago

Product/Feature summary

We are introducing a new alert box design to the health pages to highlight easy access to Cerner log-in, for patients who are Cerner patients. While there are content updates happening to the pages as well (both for unauthenticated and authenticated) those do not impact functionality.

There is a team at Cerner testing functionality. They have created test scenarios, including one that begins with our health page entry point.

Product/Feature Outline: [https://github.com/department-of-veterans-affairs/va.gov-team/tree/master/products/identity-personalization/sso] Product/Feature Epic: [https://github.com/department-of-veterans-affairs/va.gov-team/issues/9672] Product/Feature Pre-Launch Checklist: [provide link] Product/Feature Design artifacts: https://github.com/department-of-veterans-affairs/va.gov-team/issues/9693 (scroll down to "Documentation"

Product-team SMEs

[List points-of-contact (just full names) for all involved Subject-Matter domains. Most often, these would be the relevant development-team members (Product Manager, UX Designer, Front-End Developer, & Full-Stack or Back-End Developer] Product Management: Design/Research: Content/IA: Front-end: Back-end: QA: Tze-chiu Lei

Proposed user-scenarios and Staging test-accounts

[Provide specific user-senarios (all the different interactive usage-flows and/or data-specific states) to be tested. Or, link to existing issues/docs (ideally, use this Product Use Cases template from VSA Design Lead). IMPORTANT for authenticated-experiences: Staging test-accounts must be provided for all authenticated scenarios, and Staging-API test-data created to match the account-specific scenarios.]

-
- ...

How to configure this issue

[ ] Attached to a Milestone (when will this be completed?)
[ ] Attached to Epic (what body of work is this a part of? possibly Ongoing Maintenance)
[ ] Labeled with Team (vsa-authenticated-exp, vsa-benefits, vsa-benefits-2, vsa-caregiver, vsa-ebenefits, vsa-facilities, vsa-public-websites, etc.)

Definition of Done

[ ] Test Plan reviewed by Product Team.
[ ] Test Plan finalized per review feedback.
[ ] Actual Test Run(s) completed on Staging [or VSA-QA Test Run ticket created and tracked]

tlei123 commented 4 years ago

@stephorkandatx , please start building your use-case above in the ticket Description or add a link to any separate docs you my have.

Feel free to ping me on Slack as needed. Either I or @pjhill will help as needed.

stephorkandatx commented 4 years ago

From the handoff document at https://docs.google.com/document/d/11OGKfLk0fKd08pqCKT0Wvrt9ze2vZW_PSEn2kJdK714/edit#heading=h.yeejgddqil0p

Testing We have a limited number of cases in TestRails pertaining to the release of scheduling functionality for patients at the Columbus (Chalmers P. Wylie Ambulatory Care Center in MVI). https://dsvavsp.testrail.io/index.php?/runs/view/241&group_by=cases:section_id&group_order=asc&group_id=66

NOTE: As soon as we know which facilities are launching you’ll need to ask for a test credential for a user that is attached to that facility in the MVI. (There's not a good process for locating/establishing test users at a given facility. There are some VHA folks (Lynne Case, Mike Ramirez) in the #health-care-products channel in DSVA slack who can sometimes provide one. Otherwise some of our contacts in the IAM team (Damien DeAntonio, Ashley Piasecki) can try to query MVI and find a candidate record.

pjhill commented 4 years ago

Initial draft of test plan with new test case stubs created here -- https://dsvavsp.testrail.io/index.php?/plans/view/554

pjhill commented 4 years ago

Invision mockup for #5743 appears to be missing callouts for the added / changed text. I'm assuming that the text being changed is similar to the other pages that were mocked up. Confirmation of the changes to the "Medical Records" health page would be appreciated.

MickinSahni commented 4 years ago

hey @tzelei @pjhill removing this ticket from our upcoming pw sprint (26) since there are content updates pending as well as figuring out test data availability.

pjhill commented 4 years ago

hey @tzelei @pjhill removing this ticket from our upcoming pw sprint (26) since there are content updates pending as well as figuring out test data availability.

@MickinSahni, @tzelei -- Alright, the primary potential blocker regarding test data is defining at least on test user at each facility that will be launching the text / alert changes, right? Do these users also need to be associated with some kind of Cerner login as well?

Also, to check that I've synthesized the information available correctly, it seems like there are two main flows to the changes:

Unauthenticated users browsing the text changes/alerts on health care product pages
Authenticated users browsing the health care product pages

MickinSahni commented 4 years ago

Hey @kelsonic @DanielleThierryUSDSVA can you take a look at/help answer @pjhill's questions above?

kelsonic commented 4 years ago

Alright, the primary potential blocker regarding test data is defining at least on test user at each facility that will be launching the text / alert changes, right?

Do these users also need to be associated with some kind of Cerner login as well?

@pjhill We have test (fake) users that belong to at least 1 Cerner facility on staging that you can use for testing purposes, ping me and I'm happy to share the creds with you! 🙂

Also, to check that I've synthesized the information available correctly, it seems like there are two main flows to the changes:

Unauthenticated users browsing the text changes/alerts on health care product pages Authenticated users browsing the health care product pages

So there are currently 3 states:

On production environments, we show the legacy content.
On non-prod envs and when a user is authenticated AND belongs to at least 1 Cerner facility, we show the authenticated content.
On non-prod envs and when a user is NOT authenticated OR does not belong to at least 1 Cerner facility, we show the unauthenticated content.

Hope this helps! 🙂

pjhill commented 4 years ago

@kelsonic Perfectly clear explanation -- thanks very much for that! Presumably, at some point the behavior on production will reflect the scenarios that you've described in 2 and 3, right?

DanielleThierryUSDSVA commented 4 years ago

Thanks, @kelsonic! Was just pulling together the explanation of 2 and 3, but this is perfect.

kelsonic commented 4 years ago

Presumably, at some point the behavior on production will reflect the scenarios that you've described in 2 and 3, right?

@pjhill Exactly! 💯

MickinSahni commented 4 years ago

Hey @pjhill @tlei123 we're targeted 8/21 for our initial Cerner go-live. Do we have test cases wrapped up in TR? Is there anything you need from Public Websites?

tlei123 commented 4 years ago

Hey @pjhill @tlei123 we're targeted 8/21 for our initial Cerner go-live. Do we have test cases wrapped up in TR? Is there anything you need from Public Websites?

@pjhill , I'm assuming you got this, but do let me know if you need me to jump back in.

tlei123 commented 4 years ago

@pjhill , I'm assuming you got this, but do let me know if you need me to jump back in.

I'm jumping back in. @pjhill is on parental leave, returning on/after 8/26. VSP will have QA-review covered.

tlei123 commented 4 years ago

@MickinSahni @kelsonic , would one of you mind checking @pjhill 's stubbed-out TestRail test-cases please?

let me know if any steps need updating;
provide missing info:
- Entry-point URL(s);
- Staging test-account(s) for the Authenticated cases.

I'm ready to start test runs this afternoon (MT), pending above outstanding items.

Feel free to ping me for a quick Zoom mtg if you prefer. Thx much! :)

tlei123 commented 4 years ago

From Slack conversation w/ Product Team, entry-point URLs and latest copy/layout updates have been provided.

I'm updating the test-cases, and running the unauthenticated ones today.

Will need to run VSA BaM1's BDD test-cases tomorrow (Wed), but will circle back here afterward to follow-up (probably Thu 20) on test-account(s).

tlei123 commented 4 years ago

I'm updating the test-cases, and running the unauthenticated ones today.

@MickinSahni @DanielleThierryUSDSVA , turns out @kelsonic 's content updates PR for #12446 is still open right now. I've updated all my test-cases for those updates -- was hoping that PR would've been approved & merged today so I could get started on runs.

I'll circle back to this tmrw (Wed) afternoon -- after BaM1's BDD test-case updates and test-runs.

MickinSahni commented 4 years ago

👋 Tze, how do we feel about closing this out?

tlei123 commented 4 years ago

@MickinSahni , i'm finishing test-runs today, and will close this once all are done.

tlei123 commented 4 years ago

@MickinSahni , all test cases have been run.

As most of the changes are content-only, and the timeline tight here, I've deleted the cross-browser runs -- the ONLY styling changes were paragraph-spacing & text-bolding which are not prone to cross-browser/-device rendering quirks.

TestRail links

Test Plan
Coverage for References report
Summary (Defects) report [See NEW report below]

NOTE: It seems the provided Staging test-acct ssoissoetesting+mhvcss4@gmail.com may have brought up Unauthenticated versions of the content (i.e., content for Users signed-in but not belonging to a Cerner facility) in many of the Auth'd test-cases. Numerous copy mismatches were logged in the "Auth'd" defects, and it'd be advisable for @kelsonic to take a first look at them, as chances of false-positives are high.

I'm actually gonna keep this ticket Open in case we need to re-run the Auth'd test-cases [assuming we do find suitable Cerner-User test-accounts] -- I'm really not sure if the provided test-account actually enabled QA'ing the Authenticated content, except for maybe one of the 5 pages.

Feel free to reply here, or ping me on Slack, for further discussion.

cc @pjhill

tlei123 commented 4 years ago

@MickinSahni @pjhill , NEW Summary (Defects) report generated.

Turns out Staging feature-flag(s) disabled Authenticated-content for Messaging, Prescriptions, Records, and Results pages, so I've just Closed the 4 relevant Bugs as invalid, and added Retest statuses to the 4 now-outstanding test-runs in TestRail.

I'm expecting a new ticket to track when those 4 Auth'd pages will be flipped on, so that I can re-run the outstanding cases.

Feel free to reply here, or ping me on Slack, for any clarifications needed.

MickinSahni commented 4 years ago

Thanks, @tlei123. Gaining more clarity and hope to have an update on the path ahead soon.

brianalloyd commented 3 years ago

Closing per Grooming session.

department-of-veterans-affairs / va.gov-team