search

mozilla / missioncontrol

Real-time monitoring of Firefox release health
Mozilla Public License 2.0
21 stars 18 forks source link

add a crash rate for the last 24h next to the summary score #318

Open philipp-sumo opened 6 years ago

philipp-sumo commented 6 years ago

as the "current release to date" metric might become more inert the longer a firefox release cycle is going, i guess it could be beneficial to add a 2nd top-line measurement with the same calculation (geometric mean of the main+content rate for all platforms) but reflecting just the situation of the past 24h.

this might alert us of spikes/drops in the stability situation quicker, for example once we compare the rates between channel meetings...

wlach commented 6 years ago

Closely related to the discussion in #215. I feel like there is a higher level requirement (essentially "what is happening right now?") which we need to think about here.

mozillamarcia commented 6 years ago

Will - is there a way to move this into the relman_v3 bucket?

wlach commented 6 years ago

Sure thing, I hesitated because I felt like the implicit requirement stated here might just be a duplicate of #215.

I want to be careful about adding too much information to the top-level views. I understand the need but on the other hand I don't want to overwhelm people with data that is difficult to properly interpret.

One potential alternative to putting two numbers everywhere would be to add some kind of toggle between "last 24 hours" and "entire release"

mozillamarcia commented 6 years ago

I would be fine if there was a toggle - @philipp-sumo what do you think?

philipp-sumo commented 6 years ago

sure, a toggle would be fine if we would add duplicated measurements in all the places.

however i think it could also be of benefit to have the top-line summary score for the whole release and the past 24h side-by-side, as that would give consumers of the dashboard a quick indication whether things are trending up or downwards currently...

wlach commented 6 years ago

I'm not sure if you can extrapolate a trend by comparing the last 24 hours versus the entire release. It could be a useful point of comparison, but it also may not be (e.g. for Android nightly this cycle the useful point of comparison would be when 63 first branched and before the oreo sdk changes landed).