mozilla / metric-hub

Central hub for metric definitions that are considered the source of truth
https://mozilla.github.io/metric-hub
Mozilla Public License 2.0
10 stars 12 forks source link

Add OKR metrics to Desktop and Mobile products #523

Closed bochocki closed 1 month ago

bochocki commented 1 month ago

Adds OKR definitions (Engagement Rate, Retention Rate, New Profile Retention Rate) to Metric Hub (DS-3762).

bochocki commented 1 month ago

I don't know if there's a better way to do this, but I wrote a short Python script to try to validate that these metrics are working as expected after merging. The script is below.

All metrics could be queried successfully using this approach.

!pip install mozanalysis -qq
import pandas as pd

from google.cloud import bigquery
from google.colab import auth
from mozanalysis.config import ConfigLoader
from textwrap import dedent

auth.authenticate_user()

def fetch_metric_hub_definition(start_date, end_date, metric_slug, app_name, bq_project):
    start_date = pd.to_datetime(start_date).date()
    end_date = pd.to_datetime(end_date).date()

    # Set useful attributes based on the Metric Hub definition
    metric = ConfigLoader.get_metric(metric_slug=metric_slug, app_name=app_name)
    submission_date_column = metric.data_source.submission_date_column

    # Modify the metric source table string so that it formats nicely in the query.
    from_expression = metric.data_source._from_expr.replace("\n", "\n" + " " * 15)

    query = dedent(
        f"""
        SELECT {submission_date_column},
               {metric.select_expr} AS value
          FROM {from_expression}
         WHERE {submission_date_column} BETWEEN '{start_date}' AND '{end_date}'
         GROUP BY {submission_date_column}
        """
    )

    print(
        "\n",
        "\n" + "-" * 110,
        "\n" + f" Querying for '{app_name}.{metric_slug}' ".center(110, "-"),
        f"\n{query}"
    )
    df = bigquery.Client(project=bq_project).query(query).to_dataframe()

    # ensure submission_date has type 'date'
    df[submission_date_column] = pd.to_datetime(df[submission_date_column]).dt.date

    return df
for app_name in ["firefox_desktop", "multi_product", "fenix", "firefox_ios", "focus_android", "focus_ios", "klar_android", "klar_ios"]:
    for metric_slug in ["engagement_rate_v1", "retention_rate_v1", "new_profile_retention_rate_v1"]:
        df = fetch_metric_hub_definition(
            start_date="2024-03-14",
            end_date="2024-03-21",
            metric_slug=metric_slug,
            app_name=app_name,
            bq_project="moz-fx-data-bq-data-science"
        )
        display(df.set_index(df.columns[0]).transpose())