search

mozilla / experimenter

A web application for managing user experiments for Mozilla Firefox.
https://experimenter.services.mozilla.com
Mozilla Public License 2.0
113 stars 175 forks source link

Make Jestream front-end visualization more configurable, or at least richer #8938

Open ncalexan opened 1 year ago

ncalexan commented 1 year ago

In https://experimenter.services.mozilla.com/nimbus/backgroundtaskmessage-notification-release-4pct-en/results#notification_clicks I have a notification_clicks metric with three statistics: identity, sum, binomial. identity counts 1 for each client ID; sum counts 1 for each notification click; binomial gives a confidence interval of sum divided by identity: the click-through rate.

Experimenter presents this very rich information as the following:

image

It's not very useful: the CTR, which is naturally a percentage, is truncated to 0.01. The (absolute) raw numbers are not displayed. (Maybe I didn't include the correct things in the secondary outcome.)

In some way, I want a better display of this valuable information. I shouldn't have to query from the BQ tables directly to visualize this.

In my opinion, configuring this display should be part of the data definitions in the Jetstream TOML defined in https://github.com/mozilla/metric-hub. But I don't know if that's supported by the existing architecture.

How can we make the Jetstream front-end better serve consumers?

┆Issue is synchronized with this Jira Task

jaredlockhart commented 1 year ago

Thanks for filing @ncalexan ! This is very useful feedback. I'll discuss this with the data group.