Thread pool support

[ishitatsuyuki]

I'm building a stateful application, and indeed I will need separate threads to poll events. I'm wondering if a mpmc event loop would be implemented.

The current approach is separated event loop, which works well for TCP applications (the connection is managed by kernel and there's SO_REUSEADDR), but this limits threading with other patterns, like UDP networking.

[alexcrichton]

Thread pools are definitely supported! You can take a look at the futures-cpupool crate for an example of how to create a thread pool and submit work to it.

Now a different topic is to have the I/O event loop work on multiple threads simultaneously. This is a relatively rare thing to see in practice, but regardless it's currently blocked on support at the lowest level. Mio itself does not support concurrently using a Poll, which is the core of the event loop.

Does that make sense? Are you thinking of thread pools in some other fashion perhaps?

[ishitatsuyuki]

The offloading pattern seems good, but the only thing I'm concerned is the overhead (and also the additional syntax) of submission to the pool.

Concurrent polling does work at OS level but the job sync is tricky, so we'd better not do it. I was using Boost.Asio, which has thread pool built in and runs handler on all "polling" threads (it internally use mutex+condvar and only polls on one thread).

[alexcrichton]

Yes if what you're looking for is handling I/O simultaneously on multiple threads then not only does mio have to support it but tokio-core would also need synchronization and such to support it. We'd definitely want benchmarks either way to see what's what.

It's true though that we could eschew support in mio for a Mutex. I'd also want to benchmark this to ensure it's the right strategy, but may be worth looking into!

[ishitatsuyuki]

I don't think there's anything required on mio's side. For example, an abandoned project called mioco already has capability to handle multiple threads.

[alexcrichton]

Yeah it's true that we could throw the Poll itself in a Mutex, in which case we'd just be synchronizing access to epoll. That may have surprising performance characteristics on Unix, however (and may add overhead). Additionally it's not really using the power of Windows to its fullest extent as the lowest layer of IOCP handles multithreaded polling quite well!

[ishitatsuyuki]

Polling one by one in IOCP is actually more expensive; a synchronization (potentially lockfree) is cheaper than a system call.