search

Pagghiu / SaneCppLibraries

Sane C++ Libraries
https://pagghiu.github.io/SaneCppLibraries
MIT License
507 stars 11 forks source link

io_uring backend? #11

Closed ido closed 7 months ago

ido commented 7 months ago

Hi there, cool project!

Any plans to support io_uring backends for Async I/O (and event loop)?

Pagghiu commented 7 months ago

Yes it's planned. I'm just trying to understand if I can do it avoiding to use liburing as I would like to keep the library free of dependencies.

ido commented 7 months ago

The kernel interface seems fairly straightforward if you’ve worked on completion queue related settings like Infiniband (libibverbs) and other userspace networking settings. (NB: In Windows, IOCP is the closest equivalent, and in BSD, it’s kqueue.)

However, assuming you already link against a libc, I don’t see why you couldn’t optionally link against liburing and even libfabric (statically or otherwise), if the user wishes to use those backends. For most users, epoll is probably good enough, but the two approaches (completion queues vs epoll) are leaky enough abstractions that they affect how one writes their event loops in not so subtle ways.

Pagghiu commented 7 months ago

I know about IOCP or kqueue (that's how Async library is implemented on Windows / macOS).

I have no idea however about what libfabric is.

Regarding not wanting to link liburing discussion I think it's a matter of definition. In this project a dependency is anything that is not included with a standard C++ compiler / SDK / sysroot installation. A dependency is a library that must be pre-installed by a developer or compiled from its sources. This project links libc (and pthread) but libc comes with the compiler, so it's not considered dependency (it's always available). liburing instead must be installed through a package manager or compiled from source, and so by this definition it's a dependency.

I should admit however, that this is a very clear definition on macOS or Windows, that come with properly defined SDKs. On Linux doing the same is immediately more difficult once we need anything out of the libc because outside of the libc/ pthread and a few more libs you need to use some package manager to install "dev" packages (even just X11 or OpenGL / Vulkan for example) or clone and compile things with your own "build system" that I would like to avoid imposing to users.

Pagghiu commented 7 months ago

Hi @ido , I have been starting the io_uring backend. There is a first video I've been publishing on bootstrapping it

https://www.youtube.com/watch?v=YR935rorb3E

I will publish an additional one during this week getting the backend to fully pass the small test suite. I will probably cleanup the code a little bit and push it to main branch the same day when next video will be pushed out.

Pagghiu commented 7 months ago

Second video https://www.youtube.com/watch?v=CgYE0YrpHt0!

And the io_uring backend has been pushed at dc01aad0d33f94ab23fe7ad2942513f9bbb2b200 😄