Not able to generate high load

logzio / elasticsearch-stress-test

Stress test tool for Elasticsearch

Apache License 2.0

271 stars 110 forks source link

Not able to generate high load #19

Open orzilca opened 6 years ago

orzilca commented 6 years ago

Hi, great tool!

Having said that, there are some issues to be solved :)

it seems like its impossible to generate heavy load. no matter which setting I try (clients, duration, doc size), the maximum load of docs/second I can reach is 2000. i have tried setting the clients count to big numbers such as 5000, but I dont see indexing rate changes on Marvel (its never passes to 2000 docs/s). maybe I am missing something?
Missing testing of non-bulk indexing. (es.index(...))

orzilca commented 6 years ago

Actually it seems like I'm not able generate load of more than 100 docs/s. It shows 2000 because I use 1 replica.

roiravhon commented 6 years ago

Wierd, i was always able to generate load with this. But this tool was written a long time ago, and frankly, for real load testing we are using this tool which we also wrote - https://github.com/logzio/elasticsearch-benchmark-tool As it is a lot more comprehensive and can generate more "real-life" scenarios

roiravhon commented 6 years ago

Which version of ES are you testing it against?

orzilca commented 6 years ago

6.3.1

I'll take a look at the new version. There's also Rally by elastic which seems like a good benchmarking tool.

roiravhon commented 6 years ago

Yep, also a good tool. It was just too comprehensive and complicated for our benchmark needs. This tool was written for ES 2.x, so might need some adjustments for 6.x needs :)