mozilla / metric-hub

Central hub for metric definitions that are considered the source of truth
https://mozilla.github.io/metric-hub
Mozilla Public License 2.0
13 stars 13 forks source link

DS-3616 Update Metric-Hub DAU and KPI metrics #561

Closed jmsilverman closed 1 month ago

jmsilverman commented 1 month ago

For the 10 metrics in the Metric Levels Taxonomy this PR:

jmsilverman commented 1 month ago

It seems like it could be challenging to maintain these levels so they don't get stale. Is there a routine review on the levels?

I'm definitely open to suggestions around this! My idea was that the Metric Inventory list would be the most up to date and accurate and show the Last Verified Date. If something changes on one of those individual pages then the person changing it should be responsible for making sure the level tag in metric-hub is still accurate. Perhaps a quarterly or half-yearly review of all metrics is not a bad idea!

Also, it's not quite clear from this approach what the difference between the current level and the target level is. In other words how do we know when a "silver" rating is good enough, and how do we know that "this metric is currently silver but would ideally be gold"?

Again, the individual metrics pages linked to from the Inventory page are the source of truth and show both the 'current' and 'goal' levels for each metric. I considered adding "goal level" as a tag in metric-hub as well but I ended up deciding that it would be too confusing and instead went with just a single level field that represents the "current, actual level". As for the difference between the current and target levels, I see the "Future Work" row on each individual metric's page as the place to list what is needed in order to move the metric from the current level to the target level.