search

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
282 stars 201 forks source link

Analytics KPI dashboard request for VAOS #26851

Closed laurenernest closed 1 month ago

laurenernest commented 3 years ago

Request a Standard KPI Dashboard

What this form is for

Use this template to request a KPI dashboard on Domo for your product.

Additional Info

Expected delivery timelines

Your Standard KPI dashboard request will be added to the Platform Analytics & Insights team's customer support queue, and we expect tickets at the top of the queue to be pulled in every Platform Sprint Planning & Refinement (every Wednesday). We appreciate your patience.

Instructions for the VFS team

VFS-provided information

  1. VAOS Funnel Information - Full funnel for app
  2. VA Request Funnel Information
  3. VA Self-schedule Funnel Information
  4. Community Care Funnel Information
  5. Vaccine Funnel Information

Please leave the following blank

VSP Analytics & Insights Acceptance Criteria

  1. Build (See internal process document & how to)
  1. Q&A
  1. Share

Definition of Done

joanneesteban commented 3 years ago

Hi @laurenernest ! Thanks for sharing this so quickly! Our standardized dashboards are currently only supporting Forms, Content, and Search products (the dashboards shared is for Forms products, and we'll be releasing the Search & Content dashboards on July 20th). More info linked here.

The VAOS data, and potentially the design, for VAOS will be an edge case that we're preparing to support starting July 22nd after we release the other standardized dashboards. Given the additional data and design discovery, we don't have a specific timeline for when that might be completed. We apologize for the inconvenience, but please feel free to hop into one of our office hours if you have any questions about how to pull the data on your end.

CC: @thepipster @JasonTheBearded @chloedotbrown

joanneesteban commented 3 years ago

@chloedotbrown to start technical & design discovery.

chloedotbrown commented 3 years ago

Hi @laurenernest! We've updated our request template since you submitted this ticket, so following up for some extra info the older version didn't ask for.

What would you like us to use as the landing page for these 5 funnels? I see there's a "Static Landing Page" step in the full funnel – do you want that to be the landing page for all 5 funnels, or would you like to designate different ones for the other products? For the full funnel, we could also remove the landing page from Step 1 and move all the other steps up, if that makes more sense.

Let me know what you'd like to go with and if you have any questions!

laurenernest commented 3 years ago

@chloedotbrown For the 4 non-full flows, you can use /health-care/schedule-view-va-appointments/appointments/ as the landing page. That's the URL the Start scheduling button is on.

For the full flow, keep the SLP as the landing page. I want to make sure I have at least one general view of that conversion. I noticed I had the URLs in the first 2 steps reversed so I corrected that in the sheet.

joanneesteban commented 3 years ago

@laurenernest - the team is figuring out how to transform the data. We'll keep you posted on this when we have updates!

laurenernest commented 2 years ago

@chloedotbrown our latest changes to the CC flow are now live and change part of the funnel. where should I update that info?

chloedotbrown commented 2 years ago

Hi @laurenernest - thanks for letting us know! Can you update the version here? I've given you edit access, but if you have any issues getting in, let me know.

laurenernest commented 2 years ago

Updated!

laurenernest commented 2 years ago

@chloedotbrown Checking in on this. As I think about how I've been reporting on metrics and the VAOS roadmap, distinct flows will likely be the most useful for us. Since this is going to be the most complicated for you, shall we meet up again to discuss?

chloedotbrown commented 2 years ago

Hi @laurenernest, thanks for reaching out! Your timing is pretty perfect too, since the Analytics team is now fully staffed again. I've added our new Data Engineer and PM to this ticket and will set a meeting for the 4 of us to sync up on this soon. Looking forward!

chloedotbrown commented 2 years ago

Hey @laurenernest and @ldelacosta – know I mentioned following up in the #vfs-analytics Slack at the end of our meeting today, but then I remembered we have this ticket! Here are links to the resources I mentioned. If you have any trouble accessing, please let me know!

Next steps:

VAOS team

Analytics team

CC: @obrienjulie + @michelle-dooley

laurenernest commented 2 years ago

Thank you! I'll review when I'm back in the office next week.

laurenernest commented 2 years ago

@chloedotbrown I've reviewed the flows and wrote up notes on the mockup here

chloedotbrown commented 2 years ago

Awesome, thanks for these notes @laurenernest! While @obrienjulie is working on engineering approaches, I'll put together another mockup that incorporates these edits so the design is ready to go once the data is available.

laurenernest commented 2 years ago

@chloedotbrown i completely forgot about another entry point into the scheduling flows that we track with an event called vaos-schedule-another-appointment-button-clicked. It's essentially the same as vaos-schedule-appointment-button-clicked but fires only when a user goes through the scheduling flow an additional time from the confirmation page of an appointment just scheduled. i've added that event to the "Start New Appointment Flow" step in all the funnel docs.

obrienjulie commented 2 years ago

Thanks for that update, Lauren!

chloedotbrown commented 2 years ago

Sharing a revised version of the mockup that incorporates Lauren's edits. We'll need to talk to the current VAOS team to learn more about the additional metrics she requested. Worst case scenario, they won't be able to be included on the MVP but may be added later.

ldelacosta commented 2 years ago

Reviewing the mockups with @JRRoof. @chloedotbrown - let us know when you want to talk about the additional metrics Lauren requested.

michelle-dooley commented 2 years ago

Hi @ldelacosta and @JRRoof. Great to hear from you!! We have been making progress on how to track/manage the funnels. We should also be able to provide Appointment list (upcoming, past, canceled, pending) and Appointment details (VA, CC, Request) per Laurens most recent requests. I will get some time on our calendars to discuss since you are new to the team and it would be good for us all to align. Thanks!!

michelle-dooley commented 2 years ago

Hi @JRRoof and @ldelacosta - Unfortunately we are not quite complete with your dashboard. The shared steps across funnels caused some data challenges that have take some time to resolve and ensure we are counting each accurately and not overstating. This is top priority for our team to complete as I know we committed to end of March. I apologize for the delay!! We should have this to you in the next couple weeks.

JRRoof commented 2 years ago

@michelle-dooley - Thanks for the update. If we need to discuss anything, just drop time on our calendars.

chloedotbrown commented 2 years ago

Hi @JRRoof - thanks for stopping by office hours today!

As we discussed, I'm sharing links to the request forms for the VAOS Full Funnel (shared steps at start and end of all flows), Appointment Request, Self Schedule, Community Care, and Vaccines. For all 5, it would be fantastic if you and @ldelacosta could help verify:

  1. Are these current and complete? Are there any core, mandatory steps we are missing in any of these flows? If not, please add the rows we're missing.
  2. Are some of these steps optional? If so, your funnel visualizations will look clearer with these steps not included. Please indicate optional steps by changing the background color.

In addition - it would be so helpful if our team could get access to the two dashboards you showed us today on GA and in Data Studio. Could you share links/permissions so we can have a closer look? Thanks so much - if you have any questions, please reach out!

chloedotbrown commented 2 years ago

Hi @JRRoof and @ldelacosta! Wanted to follow up on these VAOS funnels☝️ and getting our team access to your Google Analytics and Data Studio reports. The reports are a nice-to-have for us, but we're waiting to get those funnels verified so we can wrap up data pipeline work on our end.

If you have any questions or there's anything we can do to help move this forward, please let me know!

JRRoof commented 2 years ago

@chloedotbrown - Last week got busy. I'll have time this afternoon and tomorrow morning to review and will get back to you with any questions. Thanks!

JRRoof commented 2 years ago

@chloedotbrown - I reviewed the funnels and the only step I found missing was on the VA Request funnel. If the Veteran is eligible for Community Care, there is a conditional page that loads and asks if they would like to request a VA appointment or a Community Care appointment. I added the step to the shared spreadsheet. Seems to me this step should be tracked, but let me know what you think.

JRRoof commented 2 years ago

@chloedotbrown - Can you send me a list of the members of your team that need access to the Dashboards? I don't own the originals, so I can't share them, but I can make a copy and share those.

chloedotbrown commented 2 years ago

Hi @JRRoof - Thanks for checking the spreadsheets! We'll take a look a the VA Request / Community Care fork you mentioned and see what makes the most sense for how to handle.

As for dashboard access, being able to see the copies would be great! I'll add emails below for folks working directly on this project. I can also help distribute, if you're able to give me Sharing permissions on the copy!

JRRoof commented 1 month ago

Working with product owners to define metrics needed monthly. Closing out this ticket.