Have any interest in a zeromq pipeline benchmark?

[lancecarlson]

What's interesting is that it seems like it sends messages about 20% slower, but it receives messages about 2% faster. This also only accounts for one level of pipelining too. Might be interesting to see how another level impacts performance.

If you're interested, I just hacked together a little prototype file here:

https://gist.github.com/lancecarlson/4670b4bbcfd1b377f70c

[tylertreat]

I don't actively use this library to run benchmarks anymore since I think it has several flaws, but I'll happily merge PRs.

[lancecarlson]

@tylertreat What were the flaws you noticed? Was hoping this was a somewhat reliable benchmark

[tylertreat]

I think it's reliable, but a few problems IMO:

• It looks at average latency
• It runs load generation on the same machine as consumption
• It runs the broker on the same machine as the client

[lancecarlson]

Yea, I can see that.. but you are still running it under the same conditions for every mq so the numbers should reflect somewhat accurate results? :)

[tylertreat]

Sure, though I think looking at average latency is dubious. Something like this which attempts to capture latency distribution is better, or you might look at Flotilla which also attempts to capture the distribution while running distributed.

You're of course welcome to use this, just interpret the data critically.