search

softwaremill / mqperf

https://softwaremill.com/mqperf/
Apache License 2.0
145 stars 37 forks source link

redpanda: low latency mode #67

Open emaxerrno opened 3 years ago

emaxerrno commented 3 years ago

hi @adamw - i saw the post (https://github.com/softwaremill/mqperf/commit/8873fa57cf554c720488b5ba5a09dfab92dd23d6?short_path=e28ac69#diff-e28ac696dc377acf5d4494af8b82c2fa6467c86cfea831bdd149c1d124eb5edf) when i was looking up redpanda projects.

We just introduced a patch for even lower latency (at expense of lower throughput): https://github.com/vectorizedio/redpanda/commit/1b0f027dc0f61fbdbf06bf77440a1e863cc2e87e

but also, the post talks about gp2 as the ebs volumes. In this setup (while I think is good to keep setup the same) you will be bottlenecke'd by IOPs, not by throughput. In fact is a bit disadvantageous to redpanda in that mode if you care about performance in that we issue a bit higher IOPs. For example, in order to gain performance improvements on local NVMe ssd devices we leverage sparse file system support ( dispatch of iocb in order 1, 2, 3, 4 but reap events from the kernel in order of 3, 2, 1, 4 - as one example)

Just wanted to add the 2 cents there.

Let me know if you want to brainstorm help to tune your setup on vectorized.io/slack always happy to help folks benchmarking.

emaxerrno commented 3 years ago

One last thing!, next wednesday we'll release v21.8.1 has a bunch of perf improvements too.

adamw commented 3 years ago

Thanks for the feedback! I think I did monitor IOPS for this test as well, but I don't have a hard record of it, so I can't be 100% sure. We should probably add this metric to the data we save for each test :)

We periodically re-run the tests with newer versions, I'll let you know here when this happens.