Roll VA Health Chat link out to 20% of landing page visitors and monitor

[wesrowe]

Description

User story

As a Cartography team member, I want to increase the percentage of MHV landing page visitors who see the VA Health Chat link to 25% (from zero), so that we can monitor for issues.

Notes

• Ideas for analytics to watch for (Team to discuss):
  • VAHC link gets some clicks
  • GA: Link should be clicked approximately once per session (for those who click it at all)
  • DD: no frustration clicks
  • Hugo: traffic to VA Health Chat with VA.gov as the referrer
  • Hugo: page views of "service recovery page" (i.e., SSOe failures) with VA.gov as referrer

Possible tasks:

• Decide what analytics to watch and what values we anticipate as successful/normal

Acceptance criteria

• Confirmed timing of increase with Hugo Padilla
• NEW: Notified Dr. Josephs of 20% rollout: meredith.josephs@va.gov
• Increased percentage for feature toggle to 20%
• Stayed in touch with Hugo for daily analytics updates on his side
• Monitored Medallia feedback for signs of trouble for several days
• Confirmed Analytics are as expected (see above)
• Ticketed follow-up analytics and/or roll-out steps

[LynneJohnson224]

8/7/2024 Refinement Notes: Potential to inject into Sprint 37, need to answer question about analytics - do we need to record link clicks in a way that's different than OOB functionality? What are frustration clicks and how are they monitored in DD (is the metric needed)? Analytics to monitor: clicks on the Health Chat link (we currently have a record event on all LP links)

[wesrowe]

@LynneJohnson224, the "frustration clicks" monitoring is under "ideas," so definitely not binding if the team doesn't consider it important/relevant.

[LynneJohnson224]

86576 is blocked - TBD on when it will get resolved. Moving this ticket back to tne Next Sprint column until 86576 is resolved.

[wesrowe]

After Hugo shared some early stats from the new service recovery page being in prod, Patrick's latest opine on Teams (10/2):

I mean it is relatively low (~2% using rough numbers) but still pretty significant (we aim for well under 1%). That said, I'm good with launching the link on VA.gov/my-health to small initial increment conditioned on Padilla, Hugo getting access to that report (assuming it is feasible) so we can closely track things together. Deal?

Hugo is still (as of 10/4) trying to get access to the analytics himself. His analytics person was OOO last week.

• It's not clear if referrer is available in pageview data on the svc recovery page
• the 2% reading may have been driven by testing right around the prod launch

The new "service recovery page" is at VA Health Chat Service Unavailable | VA Mobile – any user who views this page was trying to reach VA HC from one of the prod links; today, that would be the link on MHV Classic.

[wesrowe]

Good news! As of 10/11 Hugo has access to the following:

1. CirrusMD will provide an update on login successes, failures daily to VA Health Chat for any Veteran attempting to use VA Health Chat (vs monthly)
2. We will receive on a daily basis web reports (vs. monthly) that inform us where users are coming from to VA Health Chat (including the new VA Health Chat link on VA.gov)
3. We will receive on a daily basis web reports that inform us when users are redirected to the service recovery page

[wesrowe]

Timing: Hugo is OOO next week, so we will launch to 20% on 10/21.

[wesrowe]

Flipped to 20% ~4:30pm ET on 10/21, after

• a quick smoke-test from Kaitlin
• confirmation from Hugo that he's back in the office and ready to request analytics tomorrow

[wesrowe]

Decision was made by Robyn to up to 50% as of today 10/28. Closing this ticket and will note further monitoring in the next ticket.