I.L.: Design analytics scheme for post-launch monitoring and file Analytics implementation ticket

[wesrowe]

Description

User story

AS A PO/PM I WANT to instrument the IL app with analytics to evaluate outcomes SO THAT we can be responsive to Veterans' experiences with the app post-launch.

Mural analytics flow.

Behaviors we want to monitor

Funnel from start to finish

• Continue buttons don't need unique GA events, as funnels will be defined by URL
  • MichelleD said there is a standard form dashboard in Domo where this stuff can be tracked OOB, based on step URLs I think

"Learn" secondary funnel/success

• Do we need to put custom GA events on the Learn links to differentiate from other links on page? (probably not for regular GA, but maybe for Domo dash?)

Accidental previous-year starts I.e., click "past year" action link but then select current year from dropdown

• GA: track what year is selected

Edit flows

• Editing year could reveal an issue – the Review Info screen in the past-limits flow is where the rubber meets the road; year selected is 1 greater than the year the zip and dependents are from.
• What Veteran impact are we trying to watch out for? there isn't any new information on the Review screen that would give a user any reason to edit except that they mis-typed.
• "Back" flows may also be an indicator of confusion, how some users may "edit" instead of using those links; is there a way to see browser-back events?

Opening multiple accordions

• GA: need to label accordions in some way that is unique; could number (1 2 3) or name after threshold (up to Pension, up to NMT, etc)
• need to lose "expand all" button for this to be easier to see (will be ticketed)
• does it indicate a UX problem though? curiosity is a natural behavior we saw in research

Analytics considerations

Quality / testing notes

Acceptance criteria

• [x] Identified analytics to monitor for key outcomes and KPIs per product outline
• [ ] Drafted design for Domo dashboard
• [x] Analytics implementation ticket filed with Platform Analytics team (if necessary)
• [ ] Custom events documented, including event action and event label
• [ ] Ticket(s) created for PW dev effort around analytics

[wesrowe]

Submitted Plat Analytics implementation ticket today.

Draft designs of Domo dash and funnels are described there; need Michelle Dooley's feedback to finalize.

Draft entered in that ticket:

• What interactions need to be tracked? NOTE: Design System Components are automatically tracked

  • This product is like a form flow, and we would like to track the funnel in DOMO

  • Primary funnel URLs (on top of root url which is soon to be confirmed):

    • [root url] is starting point of flow
    • [root url]/year - NOTE: there are two flows (current year and previous year), and this url is only present in the past-limit flow. I need help in how to think about representing these two flows in a DOMO context.
    • [root url]/zip
    • [root url]/dependents
    • [root url]/review
    • [root url]/results
    • Event: open accordion (this is complicated, see "other interactions" below)
    • va.gov/health-care/apply/application/introduction (existing page) = primary funnel ending

  • Past-year funnel (maybe separate, per /year note above)

  • Secondary funnel – we would like to track clicks on a group of secondary links which we think of as a "Learn more" path, for Veterans who are not yet ready to click "Apply now."

    • https://www.va.gov/health-care/eligibility/
    • https://www.va.gov/health-care/copay-rates/
    • https://www.va.gov/health-care/get-reimbursed-for-travel-pay/
    • https://www.va.gov/health-care/health-needs-conditions/mental-health/
    • https://www.va.gov/health-care/about-va-health-benefits/

  • [Insert other interactions - please review the Design System Components and GA Data Dictionary to make sure these interactions aren't already tracked]

    • Opening of accordions on the /results screen
    • There are 4-5 accordions here. We don't care which ones people open, but we're interested in how many they open. Our hypothesis for the happy path is that users identify a single income range and open it. But we would like to track how many accordions each user interacts with, in aggregate. I need your help identifying whether each accordion needs a unique GA event, with the goal of representing the "number of accordions interacted with" in Domo.
    • There are "edit" flows initiated by links on the /review screen, which enable the user to return to an input screen to change the input.
    • Editing is a pattern we want to monitor in DOMO - I need your help figuring out how best to track it and how to represent it in Domo. (It's sort of like a funnel pattern you could track with URLs, which would also catch use of the back button. We could also instrument the edit links with unique identifiers so that they can be disambiguated

[wesrowe]

This is going to carry over into S87, as PW analytics work will depend on Platform Analytics designing their side and making recommendations back to us.

[jilladams]

Analytics team has put this in their backlog for a new team member starting in Q3. Removing from sprint for tracking until we're unblocked.

[jilladams]

current dashboard: https://analytics.google.com/analytics/web/#/dashboard/8N5ZyysFQReJn0_B7a2nUA/a50123418w177519031p176188361/