search

mozilla / missioncontrol

Real-time monitoring of Firefox release health
Mozilla Public License 2.0
21 stars 18 forks source link

Validate summarization algorithm #144

Closed wlach closed 5 years ago

wlach commented 7 years ago

The measure summarization algorithm works like this:

https://github.com/mozilla/missioncontrol/blob/4b00e594211cdef0d3f0637386331bd49a1a57ed/missioncontrol/etl/measuresummary.py

Can we do any better than this? Both in terms of the data we're sampling (different interval?) and what value we're taking from it (should we use the median/50th percentile or something else).

One obvious problem with this is that the measures gathered for the previous versions are not directly comparable with that of the current version, although I think it's probably better to have some point of comparison than none at all.

@chutten -- do you have any ideas here?

wlach commented 5 years ago

Going to close this out, have explained and talked about my methodology with lots of folks since this was written. Really should document this somewhere, but that's another story.