io_uring support

[Jonas-Heinrich]

Feature Request: io_uring support

I am not sure whether this has been asked before (could not find it via search), but has an optimized networking stack using Linux's io_uring kernel interface been considered? E.g., based on the analysis in the paper of Durner et al. [1], I would expect up to 30% reduction in CPU usage:

Use Case

Looking at an internal benchmark, we noticed that a non-trivial amount of CPU usage occurs when we just download data from S3. Besides decryption, this workload also copies a lot of data between kernel and user space (which could be reduced with io_uring).

Additionally, AWS requests could be batched to invoke only a single syscall.

Proposed Solution

Reference implementation (C++): https://github.com/durner/AnyBlob

Acknowledgements

• [ ] I may be able to implement this feature request
• [X] This feature might incur a breaking change

A note for the community

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue, please leave a comment

[1] Durner, D., Leis, V. and Neumann, T., 2023. Exploiting Cloud Object Storage for High-Performance Analytics. Proceedings of the VLDB Endowment, 16(11), pp.2769-2782.Looking at e.g. this paper's figure 12

[rcoh]

This work would be in the underlying IO libraries (Tokio, Hyper, etc.) and not in the SDK itself. I agree this is an interesting direction though!

[Velfi]

I'm going to close this since we won't be implementing it ourselves. If you find a Rust client that makes use of this optimization and have trouble getting it to work with the SDK, let us know.

[github-actions[bot]]

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.