Analytics Product KPI Dashboard for Facilities -> VAMC

[mmiddaugh]

Request a KPI Dashboard

What this form is for

Use this template to request a KPI dashboard on Domo for your product.

Instructions for the VFS team

• [ ] For additional info, read through the How to Request a KPI Dashboard Documentation
• [ ] Your KPIs are clearly linked back to Veteran outcomes
• [ ] Link your KPIs below (KPIs should be in your Product Outline)
• [ ] Submit this ticket as an epic
• [ ] Add your team label to follow progress

VFS Information

KPIs (WIP-> brainstorming ideas) @davidconlon

Areas of focus

• Locations
• Health services
• Calls to action
• Operating status
• Left Nav

Locations

• Acquisition for system page
• Acquisition for facility pages

Health services

• Navigation between system and facility within a particular health service (do veterans understand the relationship and is this helpful?)
• Use of spotlights
• Accordions opened per user session (are they opening accordions? finding what they need in the first accordion or ping-pong?) – both on system /health-services/ and on each location page
• Wayfinding -> are they scrolling up and down the page as if looking for a particular service?
• Clicks on links within health service (is: care coordinator)
  • Traffic to Care Coordinator Pages & subsequent action taken
• Actions following clicks to health services pages: Are they clicking phone numbers? Navigating away? Scrolling down?

Calls to action

• Number of clicks on "Make an appointment" CTA
  • Number of appointments scheduled through this funnel
• Number of clicks on "Register for Care" CTA
  • Number of patients registered through this funnel
• Use of “Manage your health online” calls to action
• Use of accordions on “Prepare for your visit” for Facility Page
• Use of Calls to action on Facility Page
• Number of clicks on "Pharmacy" CTA (from health services page & Facility Pages)
  • Number of prescriptions completed through this funnel

Operating Status

• Use of Anchor links (is the order of page correct?)
• Scroll depth
• Click on call-to-action in alert for “get updates on affected services and facilities

Other

• Veteran satisfaction scores (aggregate across all VAMC product)
  • scores by VISN/system/facility (?) -> TBD
• Most frequently visited pages and path to each?
• Contact us clicks on accordions
• Contact us mobile clicks on phone numbers
• Use of Anchor links (Are these effective? Which ones are heavily used and is the page organized in the right way?)

Please leave the following blank

---

VSP Analytics & Insights Acceptance Criteria

Issue 1 - Groom Analytics for [team]

• [ ] Groom analytics for technical feasibility
  • [ ] Document data sources for each metric
  • [ ] Document unavailable vs. available metrics
• [ ] If any, add implementation issue for additional necessary implementation

Issue 2 - Draft PST Dashboard for [team]

• [ ] Draft dashboard on Domo
• [ ] Pipe available metrics to Domo
• [ ] Add appropriate team members to dashboard

Definition of Done

• [ ] KPIs have been groomed and connected to Domo
• [ ] Dashboard has been shared back with the VFS team

[mmiddaugh]

@mmiddaugh to assign to bsmartin, amycesal, joanne and jon when ready

[jonwehausen]

Thanks @mmiddaugh !

[jonwehausen]

@bsmartin-ep Linking our view setup ticket #15000 as prep work for reporting / Domo dataset / kpi dashboard setup

[bmcgrady-ep]

@mmiddaugh - Is it okay if we move this to our backlog while you work on the draft for the KPI dashboard request?

[mmiddaugh]

@davidconlon to review the week of Nov 23

[mmiddaugh]

Ready for your review @bmcgrady-ep

[joanneesteban]

Since VAMC is a..very large product, breaking out the product into these sub-topics makes sense.

For the KPI dashboard, this looks like we should be reconfiguring this dashboard to be a bit more like our Covid dashboard/it should look a bit different than our other KPI dashboards, but can serve as an example of how we'll lay out the parent products with their features.

At a high level, let's take Michelle & Dave's sub-topics as sub-pages. Given the depth of this dashboard, we may need to break out the work into separate issues.

Dashboard Layout (wireframe to come): (@mmiddaugh let us know if this overall view works or should be modified for you!)

Parent Page

VAMC KPIs

• Section Title: How Veterans are finding VAMC pages
  • Chart title: System & Location page entries
  • Chart: Line graph by week of page entries to Location pages.
  • Data: Combination of page entries for system pages + location pages.
  • Chart title: Total 'Contact Us' clicks
  • Chart: Line graph by week of 'contact us' clicks.
  • Data: Combination of 'contact us' clicks on web + mobile.
• Section Title: Ease of use for Veterans on VAMC pages
  • @mmiddaugh - do you have an average user journey for VAMC pages? We may be able to construct a "time to complete" for VAMC pages, but need a bit of discovery on the business logic there.
• Section Title: How Veterans are receiving services on VAMC pages
  • Chart Title: Total Call to Action Clicks
  • Chart: Line graph by week of Call to Action clicks.
  • Data: Number of call to action clicks (outlined in Michelle's information above).
• Section Title: How satisfied Veterans are with VAMC pages
  • Chart title: Average CSAT score
  • Chart: Bar chart by week of CSAT scores.
  • Data: Average VAMC CSAT scores from ForeSee or Medallia.
  • Chart title: VAMC vs. All VA.gov Average CSAT Scores
  • Chart: Bar chart by week of VAMC scores vs. the rest of VA.gov scores.
  • Data: Average VAMC CSAT scores vs. average VA.gov CSAT scores from ForeSee or Medallia.
  • Chart title: VAMC vs. All VA.gov Total CSAT Scores
  • Chart: Line graph by week of total VAMC CSAT scores vs. rest of VA.gov total scores.
  • Data: Total VAMC CSAT scores vs. total VA.gov CSAT scores from ForeSee or Medallia.

Sub Pages

• See Michelle's outline in the body of the issue. We can break this out more as we start.

@mmiddaugh would you be able to elaborate on the following metrics?

• Acquisition for system page
• Acquisition for facility pages
• Use of spotlights

[mmiddaugh]

@joanneesteban

Regarding Acquisition for system/facility pages

• We're looking for traffic volume but also understanding about the relationship between the system and facilities -> How do they come to the pages? Are they accessing the system from a facility page (or vice versa)? Or arriving there organically from Google/search? do Veterans visit both the system and facility? Do they navigate from one to the other? (Does that help?)

---

Regarding Spotlights

• Number of clicks on links featured in the spotlight area (system vs facility). The links are the same at both the system or facility level. We're interested in knowing if:
  • Veterans access the relevant page by clicking on the link in the spotlight area (vs a different navigational path)
  • Access through system page or facility page?

System spotlight on Pittsburgh home page Facility spotlight on Pittsburgh VA Medical Center University Drive page

Spotlight screenshot


[joanneesteban]

@JasonTheBearded when you get the chance, please feel free to pull this into your queue & let the team know if you need support since there are a bunch of metrics here.

[JasonTheBearded]

@mmiddaugh Can you provide all the pages you want tracked in the dashboard please? Thanks! Looking for the URLs in particular

[mmiddaugh]

Hi @JasonTheBearded that's a complicated question! There will be 144 health systems rolling out over the next 6 months and we'll want to be able to get metrics on them once they're live. For now, only one is live on Production.

System home page

• https://www.va.gov/pittsburgh-health-care/

Facilities pages

• https://www.va.gov/pittsburgh-health-care/locations/pittsburgh-va-medical-center-university-drive/
• https://www.va.gov/pittsburgh-health-care/locations/h-john-heinz-iii-department-of-veterans-affairs-medical-center/
• https://www.va.gov/pittsburgh-health-care/locations/beaver-county-va-clinic/
• https://www.va.gov/pittsburgh-health-care/locations/belmont-county-va-clinic/
• https://www.va.gov/pittsburgh-health-care/locations/fayette-county-va-clinic/
• https://www.va.gov/pittsburgh-health-care/locations/washington-county-va-clinic/
• https://www.va.gov/pittsburgh-health-care/locations/westmoreland-county-va-clinic/

Navigation within pages with the https://www.va.gov/pittsburgh-health-care/ root

[mmiddaugh]

Next to roll out is va.gov/erie-health-care/

[JasonTheBearded]

@mmiddaugh what is the start date for the Pittsburgh health system? Trying to see how far back the dash needs to go. Thanks!

[joanneesteban]

@mmiddaugh hi! Also bumping this "ease of use" question: do you have an average user journey for VAMC pages? We may be able to construct a "time to complete" for VAMC pages, but need a bit of discovery on the business logic there.

[mmiddaugh]

@joanneesteban my apologies for the delay Pittsburgh health system was "soft-launched" in September 2019 (pre-Facilities team involvement). I don't know that there was much Veteran use at that point so I don't think it is necessary to go back that far for data. I'm looking into the "average user journey" question and will try to get you the answer by EOD

[mmiddaugh]

@joanneesteban I can only guess about the flows around the home page calls to action (such as clicking the "Make an appointment" CTA -> on the home page to the Make an appointment page to the various locations from there).

Outside of those, I don't think the "typical user flow" is known.

The history of these pages is that no one was going to VAMC pages - Veterans didn't know they existed and if they did know - the pages were out of date, and none of them pointed to online tools.

If anything, we might need the analytics to tell us the typical user flow.

[mmiddaugh]

one last thought, @joanneesteban our expected user flow is google/search to facility page (not to system or through system to facility)

[JasonTheBearded]

@mmiddaugh did you mean to close this ticket?

[mmiddaugh]

@joanneesteban Dave and I refined our top priority VAMC Dashboard metrics:

1. Facility health service accordions
   • which health services accordions are opened the most often? how many accordions are opened per user session?
   • To inform if changes are needed in the sort order of the accordions and to determine if Veterans find what they need or have to ping-pong among multiple services
2. Calls to action
   • Number of clicks on "Make an appointment" CTA/Number of appointments scheduled through this funnel
   • Number of clicks on "Register for Care" CTA/Number of patients registered through this funnel
   • Number of clicks on "Pharmacy" CTA/Number of prescriptions completed through this funnel
3. Wayfinding/scroll depth
   • to see if they are scrolling up and down as if looking for a particular service

[joanneesteban]

Thanks, @mmiddaugh ! This is all really helpful!

[joanneesteban]

@mandy-uxd : we could use your design help, here! @JasonTheBearded has the metrics prepared 🎉, and we're thinking that those top 3 metrics can live on one "parent" dashboard. But since there are so many VAMCs (VA Medical Centers), we're worried about this getting very convoluted.

In terms of dashboard org structure, I'd like to start using our framework, & keep it at the Framework level where we can:

• Ease of use: How easy was this for Veterans to use?
  • Accordion clicks/Navigation
• Service completion: Are Veterans successfully completing their tasks
  • CTAs

Wayfinding/scroll depth: we've bucketed this under Service completion, but I think in this case it's more Ease of Use (How easy was it to use/how did they use it), since these aren't pure content pages.

[mmiddaugh]

This is satisfied by the DOMO Content dashboard