zesterer / flume

A safe and fast multi-producer, multi-consumer channel.
https://crates.io/crates/flume
Apache License 2.0
2.47k stars 85 forks source link

Update benchmarks to reflect current performance #114

Open TimonPost opened 2 years ago

TimonPost commented 2 years ago

Would be nice to see the current benchmarks in readme being updated to how performances matched the current state of things. Thanks!

Boscop commented 1 year ago

Is flume still faster than crossbeam-channel? These benchmark diagrams say different things: https://github.com/zesterer/flume#benchmarks https://github.com/crossbeam-rs/crossbeam/tree/master/crossbeam-channel/benchmarks#results

zesterer commented 1 year ago

I can only speak to the results I've found locally, on my machine: mileage may vary. If someone feels like they have the time to investigate this further I'm happy to update them. Note that the version of crossbeam used in the benchmarks on the README is a few years old at this point.

The benchmarks we have inside the repository (which are not crossbeam's benchmarks shown in the screenshots) are fairly easy to run and might give you a decent picture of things. They're quite extensive and seem to show flume drawing roughly even with crossbeam in most situations, with crossbeam sometimes pulling ahead (but never more than by a factor of 2-3, even in the worst case).

Edit: One thing I will say though: although crossbeam is undoubtedly a brilliant piece of software engineering, I have never personally been able to recreate the sort of numbers they present, even across 2-3 different consumer machines. Perhaps I have the wrong setup, I don't know: I've never benchmarked flume on a HPC cluster or the like. crossbeam's benchmarks do seem to be awfully narrow though, not covering all that many useful cases.

Edit2: I just did another run of our benchmarks locally using the latest version of crossbeam and flume to see if anything had changed. Here are the numbers, for anybody interested (I've added gaps between each group of apples-to-apples benchmarks to make things easier to read).