Alternative callbacks

[ghost]

Hello,

Problem description goes like this:

The APIs of MIO differs a bit from those of libuv and libev and similar. The main problem is that MIO requires the calling thread to dispatch events "as it sees fit" rather than having a way to register callbacks that are called by MIO itself, according to some documentation.

---

For instance in libuv you create polls, uv_create_poll and attach a callback to them along with what they should poll for. Whenever the event loop fetches that readiness from the system, it (the event loop) automatically dispatches them by calling the callback.

With MIO, you just get an iterator and YOU, THE APP dispatch them based on this "token".

This means there is a problem integrating with this MIO event loop since there is no way for an outsider to hook into the MIO event loop without also controlling the app that uses MIO. This is not a problem in many other event loops (because of the callbacks).

What about adding a very simple alternative way to attach callbacks to polls? You already have this concept of "tokens" so what if the token is a callback and you just call it? When you iterate over the events, a simple check if the token is a callback and if so, call it.

[Darksonn]

My first two initial thoughts are:

1. This can be implemented on top of mio.
2. Callbacks are usually a terrible user experience in Rust because you need to use Rc/RefCell or similar all over the place.

[ghost]

This can be implemented on top of mio.

Absolutely. But supporting some kind of callback poll (as an optional alternative - it can literally be just a separate callback poll) unlocks a more "standardized" MIO event loop.

I'm absolutely not proposing changing how most users use MIO, I'm only proposing the addition of an alternative, callback-dispatched poll type.

That way one can chain and integrate and pretty much do whatever together with MIO without having to hook into Tokio and whatnot atop. In fact looking at Tokio, I don't really see anything that would solve this.

[Darksonn]

Tokio deliberately does not allow you to talk directly to the underlying mio instance, since it would otherwise be a breaking change if Tokio upgraded the version of mio it uses. Instead, Tokio provides AsyncFd.

[Thomasdezeeuw]

Hey @alexhultman I have to agree with @Darksonn here. Mio's design is fundamentally different than that of libuv/libev. Mio is small wrapper around epoll/kqueue, which exposes roughly the same model as Mio. Adding callbacks on top of that is possible, but as @Darksonn already mentioned, this is also possible to do outside of Mio. So, right now I don't really see a case to add callbacks to Mio.

I'm curious about your use case. You mentioned a ``"standardized" MIO event loop'', but I don't think this is really a goal on it's own.

[ghost]

Pretty much all "event loops" I've seen (more than 5) have callbacks. This allows one to register (third party) FDs and handle them separately from the rest. This is important for seamless integration with other event-loops (for instance you can poll for ready events on the epoll_fd itself) coming from other languages or "stacks".

AsyncFd looks to be pretty much this, however now we are at a completely different abstraction level and it seems like AsyncFd comes with its own co-routine which means there are a lot of added "opinionated" stuff atop it which makes it impossible to really integrate without entirely changing how third-parties work fundamentally.

libevent, libev, ASIO, libuv, and many others expose callbacks and a "run method" where the event loop runs and dispatches callbacks on its own. Again this allows third-parties who are not familiar och dependent on or aware of how the dispatching works, to still integrate.

The use case is integrating foreign languages and code which is not written against Tokio but rather the standard Unix syscalls to work with MIO. All that is really needed is a way to poll for an FD with callbacks.

[Thomasdezeeuw]

Pretty much all "event loops" I've seen (more than 5) have callbacks. This allows one to register (third party) FDs and handle them separately from the rest. This is important for seamless integration with other event-loops (for instance you can poll for ready events on the epoll_fd itself) coming from other languages or "stacks".

There are many different designs for event loops, using callbacks is just one. It's a popular one, mainly in the C/++ language, but Mio is written in Rust. As such, we have a different design. Such a different design in fact that Mio doesn't even provide an event loop. It provides utilities for handling async I/O, such as networking types, and a way to poll the readiness of these types, that's it. If you want to create an event loop with callbacks on top of Mio that it's possible. This is what Tokio (and Heph) more or less are, except they use Futures instead of callbacks.

AsyncFd looks to be pretty much this, however now we are at a completely different abstraction level and it seems like AsyncFd comes with its own co-routine which means there are a lot of added "opinionated" stuff atop it which makes it impossible to really integrate without entirely changing how third-parties work fundamentally.

libevent, libev, ASIO, libuv, and many others expose callbacks and a "run method" where the event loop runs and dispatches callbacks on its own. Again this allows third-parties who are not familiar och dependent on or aware of how the dispatching works, to still integrate.

I don't know ASIO, but the other are all written in C and use the same (similar) design of callbacks. If you want to support third-parties you can use the Registry type to allow third parties to register themselves, where your application hands out the Tokens so you know what function to call (back).

The use case is integrating foreign languages and code which is not written against Tokio but rather the standard Unix syscalls to work with MIO. All that is really needed is a way to poll for an FD with callbacks.

This all possible with Mio. It's just that this isn't the goal of Mio. You shouldn't see Mio as a complete replacement of libuv/libev, but a replacement of kqueue/epoll on top of which you could build something like libuv. In return Mio is smaller, doesn't spawn thread and you remain in control of the event loop.

[Darksonn]

The use case is integrating foreign languages and code which is not written against Tokio but rather the standard Unix syscalls to work with MIO. All that is really needed is a way to poll for an FD with callbacks.

Just to be clear, AsyncFd expects you to call the read/write Unix syscalls yourself. If you want to implement callbacks on top of Tokio, you can do that like this. This implementation expects the callback to read/write from the socket using the ordinary unix syscalls until it returns EWOULDBLOCK.

[ghost]

You shouldn't see Mio as a complete replacement of libuv/libev, but a replacement of kqueue/epoll on top of which you could build something like libuv

Yeah I'm starting to accept this - this is what I feared from the start..

Mio is written in Rust. As such, we have a different design

This is irrelevant. Computer science is computer science regardless of brand.

If you want to implement callbacks on top of Tokio, you can do that [..]

Yes, sure, of course. But then you depend on Tokio which is an entire different beast, and you have these Futures enforced on you. That's way too high level for my integration to make any sense. A callback is 8 bytes, FD is 4 bytes - these Futures are way too dependent on Tokio and way too "opinionated" if you understand how I mean.

Well, thanks for the hints of AsyncFd anyways, but I can't make that work. Difference in abstraction for MIO and Tokio is just too massive and I'm not interested in building anything atop Tokio.

[Thomasdezeeuw]

I'm going to close this as I don't think we're going to support callbacks as it can be implemented on top of Mio.