search

goadapp / goad

Goad is an AWS Lambda powered, highly distributed, load testing tool
https://goad.io
MIT License
1.88k stars 190 forks source link

replace average times with percentiles #12

Closed ghost closed 7 years ago

ghost commented 8 years ago

Average response times can be misleading with high standard deviations, so I would suggest using either percentiles or mean with percentiles. I'd also add the max response time.

[mean, 50, 95, 99, max]

wendigo commented 8 years ago

I can try to implement this @sdsykes

wendigo commented 8 years ago

Well, it won't be easy change as I would like to introduce hdr histogram both into lambda, queue and aggregations.

ghost commented 8 years ago

I'd probably forget mean:

[min, 50%, 75%, 95%, 99%, Max]

zeph commented 7 years ago

@wendigo nice to see you too on this project ;) ...cheers from OLX/NaspersClassifieds@Berlin

zeph commented 7 years ago

implemented & merged

zeph commented 7 years ago

@aidylewis please, have a try

ghost commented 7 years ago

@zeph Off work today, but will test tomorrow. Thanks.

subwiz commented 7 years ago

This is not in master? This commit seems to be available only in issue#37 branch.

zeph commented 7 years ago

@subwiz I merged https://github.com/goadapp/goad/pull/101 already... I don't get it and issue#37 's branch is merged too https://github.com/goadapp/goad/pull/99

...can u pinpoint me the commit u refer to? or make a pull request and I'll review it

subwiz commented 7 years ago

I was specifically looking for the -i parameter to compute the percentiles. That seems to be not available in the master?

zeph commented 7 years ago

it's not in the ab (apache benchmark) command line, so I'm asking our contractors to first baseline on that interface and add the extra features as part of a file based configuration (it will include a lot of other things) ...coming soon ;) [a month or two]

subwiz commented 7 years ago

Thanks @zeph! Till then we are building branch issue#37 for the percentile computation.

palmerabollo commented 7 years ago

The change that shows latencies in issue#37 corresponds to goad 1.2.0. That version is already very different than master/2.x. It would be a pity to lose this change because averages are often misleading. Would it be possible to incorporate that change back into master?