Allow to switch async runtimes, possibly with ability to use io_uring

[drogus]

I've been playing with io_uring lately and I'm amazed at performance it can achieve for I/O operations. I'm also really interested in using lunatic for high performance applications that could also benefit from a sandboxed environment.

At the moment there are few options to run async code using io_uring: glommio, monoio and tokio-uring (although tokio has only the File API implemented). I'm not sure if there are any immediate plans to implement io_uring in async-std.

I would be interested in changing the lunatic code to allow using a different runtmie, most probably using features. There are a few problems, though:

1. I'm not sure if this is something that the maintainers would want? I think it's valuable even for tokio compatibility, but it comes with a more complex code. I don't want to start writing such a major change without an OK from maintainers, so I wanted to ask first.
2. How hard would it be to do it? I quickly skimmed through the code and it doesn't seem like there is a lot of places that would need to be customized, but that's more of a guess than informed opinion.
3. Would it even be possible to plug thread per core runtimes like monoio? As WASM modules are isolated my guess would be that it's possible, but again, I'm not familiar with the code

[teymour-aldridge]

There's also https://docs.rs/rio/latest/rio/ for io_uring.

[bkolobara]

Having the flexibility to switch async runtimes would be nice. This could also lead to different default runtimes depending on your OS. For example, on linux we use io_uring, but on windows we stick to async-std/tokio. That said, here are some issues I can see with this approach.

Rust/cargo doesn't have support for mutually exclusive features. Being able to select a runtime just by selecting a feature becomes a bit tricky. If a user doesn't specify a runtime in the feature set or specifies multiple, they are going to get cryptic error messages of duplicate or missing imports. Features are meant to extend the function set, but not to "choose" between multiple ones. This approach is always going to feel a bit "hacky".

Changing the runtime has also deeper implications on the host functions. Host functions working with TcpStreams are going to be different for the TcpStream type of async-std and the TcpStream type of monio. This could be solved by splitting out host functions into separate crates and making them dependent on the selected runtime. Splitting them out may be a good idea in general. However, these 2 types are not equal. One of them implements Clone, the other doesn't. It's a common pattern in lunatic to clone your stream give one of them to a process reading from it and the other to a process writing to it. This is impossible inside the world of monoio. Monoio's TcpStream uses a pattern of splitting streams into read and write halfs, but this would require us to expose this pattern to the guest code and introduce additional resources (ReadTcpStream and WriteTcpStream).

As you can tell by now, there are small differences in the implementations of the runtimes that make it impossible to just switch out one for another as they influence the guest API. This would require us to find a subset of features and APIs that we can express with all runtimes and shape our guest API around it. This includes some breaking changes like starting to differentiate between ReadTcpStream and WriteTcpStream. This may also lead to less ergonomic guest APIs as we need to use the "not ideal one", because not everyone supports the same subset.

io_uring currently gives the biggest performance boost to filesystem operations, but we use the wasmtime-wasi crate to expose filesystem operations to the guest. At the moment wasmtime-wasi doesn't even have support for async, all operations are blocking. They are adding async support, but it will probably not use io_uring based runtimes. To get io_uring based filesystem support we would need to implement all the WASI host functions and this is a huge undertaking.

I think the biggest drawback of io_uring is that all runtimes pin the future to a thread. Without Sync and Send futures we are not able to rebalance processes. In the case where a few heavy CPU bound processes end up spawned on the same thread will result in throttling. Even they can utilise all the cores in parallel, we can never move them and they keep fighting for the same CPU core.

There is probably a "perfect" runtime that we could build that can't move processes when doing I/O with io_uring, but can if the process is not performing any I/O at the moment. Using a work-balancing approach (like Erlang) instead the current work-stealing one could also be interesting. I still think this is a problem worth solving and getting the performance boost from io_uring on linux could be worth the work.

[wishfoundry]

small note: features can be made mutually exclusive quite easily

#[cfg(all(feature = "runtime-uring", feature = "runtime-tokio"))]
compile_error!("runtimes are mutually exclusive")