mozilla / missioncontrol

Real-time monitoring of Firefox release health
Mozilla Public License 2.0
21 stars 18 forks source link

Consider only getting data for official Mozilla releases #197

Closed calixteman closed 6 years ago

calixteman commented 6 years ago

Here: https://data-missioncontrol.dev.mozaws.net/#/beta/windows/content_crashes?timeInterval=6566340&relative=0&percentile=99&normalized=1&disabledVersions=&versionGrouping=version&startTime=1510704000

we've data for 57.0.1 & 57.0.4.

wlach commented 6 years ago

We should only be showing aggregated data from pings tagged with the "beta" channel in this view. Are you sure we did not release those versions to the beta population at any point? /cc @fbertsch

calixteman commented 6 years ago

It seems that there is a beta with version 57.0.4: http://bit.ly/2nrbuOJ I think this is noise and we should try to remove it. An idea could be to use buildhub data to keep only official releases.

wlach commented 6 years ago

Lots of discussion around this here: https://bugzilla.mozilla.org/show_bug.cgi?id=1401979

tl;dr: filtering this data using telemetry-streaming is somewhat complicated. Also IME buildhub is not yet something we can rely on 100% of the time. Maybe the easiest thing is to do something on the missioncontrol side to keep an ongoing record of "officially sanctioned releases" using data from buildhub. That way the user can ask to only receive the data which has that form, while still having the option to get everything if necessary (this is probably the only sane option for linux release, where 99% of what we receive is "unofficial" telemetry).

wlach commented 6 years ago

Increasingly convinced we should only process official releases in missioncontrol, though we need to figure out what will happen with Linux. Maybe flatpak and snappy are the solution here? Snappy builds don't seem to be on buildhub yet though, filed https://bugzilla.mozilla.org/show_bug.cgi?id=1456164 to see what's going on.

wlach commented 6 years ago

We started using only builds we find in buildhub in e60b9fdb59497ed1c9808bd2a0d7d1fa3a55e186