Analytics: Convert DD dashboard to custom metrics

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:

https://depo-platform-documentation.scrollhelp.site/index.html

281 stars 197 forks source link

Analytics: Convert DD dashboard to custom metrics #87751

Closed katbrinkley closed 3 weeks ago

katbrinkley commented 1 month ago

Description

User story

As a Cartography team member, I want to convert all the datadog widgets into custom metrics so that we can preserve this post-launch integration data.

Notes

Florence and Kat met with Mark D. and Robyn about how to do this

Possible tasks:

Attempt to use the same formula from the current queries, should work okay based on what Mark demoed
Pair with developer to make sure everything looks right and is being tracked correctly

Acceptance criteria

Create corresponding custom metrics to all widgets in the MHV Integration RUM dashboard.
Verify with team that everything was converted correctly

katbrinkley commented 1 month ago

Updates: All widgets need to be complete before creating custom metrics.

The MHV home page main clicks widget still has some clicks that aren't being tracked (Review medications, Subscribe to newsletter, and all articles). This needs to be done on the backend.

Next steps (tickets I will create)

Convert the funnels widgets into clicks (conv. or eng.) > create custom metrics (next task for me)
Dashboard the KPI widgets based off of the new custom metrics

katbrinkley commented 1 month ago

Blocked until I finish this ticket and the backend bugs are fixed (I believe Daniel has tickets for those)

wesrowe commented 1 month ago

@katbrinkley, I just noticed this didn't get a Sprint 36 tag. Do you see this as something we're still wanting to do? OR, did we kind of decide to wait and see what the evaluation plan requires?

katbrinkley commented 1 month ago

Updates: Chatted with Wes Thursday about this - custom metrics for all the widgets are created, but we're going to hold off on using those to create a whole new dashboard as it's harder than we expected and we think the evaluation plan will show what we need and if custom metrics is the right route or not.

wesrowe commented 1 month ago

Additional color on that – we tried making a widget driven by a custom metric, and the number coming out of our custom metric (the numerator for bounce) was about 2.5x the value that the same query surfaced in the RUM version of the widget. So we have some debugging to do and further learning about how to use custom metrics.