search

wyfo / pyo3-async

PyO3 bindings to various Python asynchronous frameworks
MIT License
30 stars 1 forks source link

Releasing the GIL #1

Closed ahirner closed 1 year ago

ahirner commented 1 year ago

Hi, I like the runtime agnostic approach!

Is it possible to write a Coroutine which releases the GIL, then runs some compute in a (scoped) thread? If so, how and if not what part of the API should be touched?

Similarly, I'd like to know how to push results from a GIL released thread into a asyncio::AsyncGenerator.

wyfo commented 1 year ago

Actually, the default implementation release the GIL when Rust futures are polled https://github.com/wyfo/pyo3-async/blob/9e1b480d4d62b59cedaff376d2d240b9d641c45c/src/lib.rs#L35.

To "push results", I would use a channel like tokio mpsc, and convert the channel receiver into stream, which can then be wrapped into asyncio::AsyncGenerator.

ahirner commented 1 year ago

Amazing that's already taken care of by default. Thanks for pointing to the exact impl!

wyfo commented 1 year ago

In the end, releasing the GIL don't seem to be the best default behavior. See #2 where I explain why the behavior will be reverted in the next release.

Of course, I've provided a simple way to release the GIL. I will wait a bit before publishing the next release, if you have some remarks about this decision, don't hesitate to write them.