sharded-thread
"Application tail latency is critical for services to meet their latency expectations. We have shown that the thread-per-core approach can reduce application tail latency of a key-value store by up to 71% compared to baseline Memcached running on commodity hardware and Linux."[^1]
[^1]: The Impact of Thread-Per-Core Architecture on Application Tail Latency
Introduction
This library is mainly made for io-uring and monoio. There are no dependency
on the runtime, so you should be able to use it with other runtime and also
without io-uring.
The purpose of this library is to have a performant way to send data between
thread when threads are following a thread per core architecture. Even if the
aim is to be performant remember it's a core to core passing, (or thread to
thread), which is really slow.
Thanks to Glommio for the inspiration.
Example
Originally, the library was made when you had multiple thread listening to the
same TcpStream and depending on what is sent through the TcpStream you might
want to change the thread handling the stream.
You can check some examples in the tests.
Benchmarks
Those benchmarks are only indicative, they are running in GA. You should run your own on the targeted hardware.
It shows that sharded-thread based on utility.sharded_queue is faster (~6%) than
if we built the mesh based on flume.
References
- Glommio example on their sharding
- The original monoio issue
- Sharded Queue - the fastest concurrent collection
License
Licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT) at your option.