Add benchmark for xtra vs tokio mpsc channels

[thomaseizinger]

A first implementation of a benchmark. Here are some first results:

increment/xtra/100      time:   [940.73 us 969.06 us 998.71 us]
increment/xtra/1000     time:   [9.6542 ms 9.9258 ms 10.205 ms]
increment/xtra/10000    time:   [97.921 ms 100.49 ms 103.09 ms]
increment/mpsc/100      time:   [14.864 us 14.972 us 15.076 us]
increment/mpsc/1000     time:   [144.32 us 145.35 us 146.34 us]
increment/mpsc/10000    time:   [1.3135 ms 1.3333 ms 1.3532 ms]

Fixes https://github.com/Restioson/xtra/issues/116.

[thomaseizinger]

I am guessing the major speed difference comes from the allocations. xtra performs at least two allocations as part of wrapping messages in its envelope, plus there might be allocations within the channel to store waiting senders / receivers.

[Restioson]

True - this is simply the cost you pay for dynamic message dispatch. I would wonder how it'd look if the Tokio example also dynamically dispatched message handlers

[thomaseizinger]

True - this is simply the cost you pay for dynamic message dispatch. I would wonder how it'd look if the Tokio example also dynamically dispatched message handlers

Would that be a fair comparison though? The actor as shown in the benchmark is how I'd implement it. With an enum to handle multiple cases.

[Restioson]

Would that be a fair comparison though? The actor as shown in the benchmark is how I'd implement it. With an enum to handle multiple cases.

It's comparing different things - in the dynamic dispatch case what we are really measuring is the channel performance of xtra's internal channel. Previously this could be extrapolated from benches of flume, but this is no longer the case

[thomaseizinger]

Would that be a fair comparison though? The actor as shown in the benchmark is how I'd implement it. With an enum to handle multiple cases.

It's comparing different things - in the dynamic dispatch case what we are really measuring is the channel performance of xtra's internal channel. Previously this could be extrapolated from benches of flume, but this is no longer the case

As a user though, I don't really care what xtra is using internally? I think xtra is quite appealing from an ergonomics perspective. The only thing that these benchmarks should show IMO is that they will not pay a massive price for these ergonomics. 10us per message is still pretty good and likely negligible compared to what the application will actually do.

At least in these benchmarks, the mpsc channel is ~70x faster but I think we are operating at a latency scale here where if that difference matters to you, you are probably going to write everything yourself anyway? I've never worked on applications that were performance critical on that scale so I don't actually know.

[Restioson]

As a user though, I don't really care what xtra is using internally? I think xtra is quite appealing from an ergonomics perspective. The only thing that these benchmarks should show IMO is that they will not pay a massive price for these ergonomics. 10us per message is still pretty good and likely negligible compared to what the application will actually do.

From a user perspective, it is not a useful measurement, I agree. However, it would still be useful to look at for us (the implementors) to see how fast xtra's channel is in comparison to tokio.

At least in these benchmarks, the mpsc channel is ~70x faster but I think we are operating at a latency scale here where if that difference matters to you, you are probably going to write everything yourself anyway? I've never worked on applications that were performance critical on that scale so I don't actually know.

Agreed. 9us for sending one message is negligible for most applications.