tsenart / vegeta

HTTP load testing tool and library. It's over 9000!
http://godoc.org/github.com/tsenart/vegeta/lib
MIT License
23.42k stars 1.36k forks source link

[reporter/hist] Create new feature in which histogram could be presented with exponential-sequence x-axis #600

Closed wu0h961738 closed 2 years ago

wu0h961738 commented 2 years ago

Background

This PR manages to improve the distortion of which executing command 'report -type=hist[]' causes, the distortion that, if arithmetic x-axis was applied, the result set having approximate latency might be cohered into single bucket while trivial cases, including P99 or P999, might be dispersed into trivial buckets. However, Labeling x-axis in form of exponential sequence separates the high-density result set while at the same time cohering scattered cases, making graph more readable.

Example case below both comes with 200 requests, with 20 QPS and lasting 10 seconds, presenting through exponential-sequence bucket on the left side and arithmetic bucket on the other side.

Common phenomenon in which large amount of request data are under stress testing usually causes distortion. In practice, for example, the amount of requests goes above 100,000 if test comes with 10,000 QPS lasting 10 seconds. Then, the outcome of graph would be presented with a normal distribution( or right/left-skewed distribution), meaning that 65%, even 90%, above of cases fall in the certain range, therefore causing distortion on graph. Further, P99, the other result set worth to be observed as well, could be underestimated if the amount of them, usually, turns to be scarce.

Therefore, in order to prevent such readability-relative issues, an alternative method printing histogram comes into play: exponential sequence.

Checklist