search

ftlabs / perf-widget

22 stars 2 forks source link

Add percentage difference #60

Closed JakeChampion closed 8 years ago

JakeChampion commented 8 years ago

This work is a starting point for figuring out which insights to return to the user using a simple heuristic. The percentage difference will be used to decide whether an insight is really useful to show.

Examples: "We have faster than The Guardian" -- this is useful if we are say, 30% faster but is not useful if we are 0.3% faster.

AdaRoseCannon commented 8 years ago

Saying something is slower by a % is ambiguous. Try being more explicit. i.e. It took 1.57x longer.

Also there is too much precision here. Perhaps to the nearest 1% or 5%?

AdaRoseCannon commented 8 years ago

Try to avoid having more than 2sf of data, users don't need the precision also the data is not that precise. Two specific loads were measured that precisely but if you were to do an aggregate you would probably find there is at least a 10% margin of error in the comparison. So providing that level of precision gives false confidence