search

awestlake87 / pyo3-asyncio

Other
300 stars 45 forks source link

Is it possible to support monoio runtime? #113

Open rainj-me opened 7 months ago

rainj-me commented 7 months ago

Monoio is a thread-per-core Rust runtime with io_uring/epoll/kqueue. Is it possible to support monoio runtime at pyo3 ?

awestlake87 commented 7 months ago

Short answer is yes, but not out of the box.

This library has a few generic traits that can be implemented for Monoio to provide the runtime-specific details like spawning tasks and task locals. As long as you implement Runtime and ContextExt for a type and JoinError for an error type, you can basically unlock all of the Rust <-> Python conversions. tokio and async-std support in this library were both written using these traits, so you can probably use those as reference.

I'm not 100% certain how monoio handles task locals. Basically to use some of the nicer conversions, we need some sort of task local storage to track references to Python's event loop and context vars. This is similar to thread-local storage, except each worker thread can be running several tasks, and in order to fully support Python's runtime model we need to track some state per-task. Just at a glance, it looks like scoped_tls might work? Without testing it out I can't be sure. I don't think task-locals are required, but without them the usage can be cumbersome or cause compatibility issues with Python tasks that make use of contextvars

I'm open to supporting this runtime in future versions if it's feasible!