Unable to run an python async function from a library trait because "no running event loop"

[vicky5124]

🌍 Environment

• Your operating system and version: Arch Linux - 5.14.3-zen1-1-zen
• Your python version: CPython 3.9.7
• How did you install python? Did you use a virtualenv?:

  sudo pacman -S python python-pip
  pip install venv -U --user
  python -m venv .env
  source .env/bin/activate
  pip install maturin

• Your Rust version: rustc 1.57.0-nightly (9bb77da74 2021-09-13)
• Your PyO3 version: 0.14
• Have you tried using latest PyO3 master?: No

💥 Reproducing

Is what I'm trying to do down here possible? Have a rust trait be able to call async python code in a library? I think this isn't working because it's a library, and all the examples are shown with binary code with a tokio or async-std runtime running. Or maybe Python::with_gil() isn't getting the current instance of python, but it's getting a new one?

#[pyclass]
#[derive(Clone)]
pub struct EventHandler {
    inner: PyObject, // Handler class in Python
}

// Internally, i have a method that takes an `impl InnerEventHandler` and uses
// it's methods to call events. This trait signature cannot be moddified.
#[async_trait]
impl InnerEventHandler for EventHandler {
    async fn start(&self, event: Start) {
        let slf = self.clone(); // 'static borrow zzz

        let future = Python::with_gil(|py| {
            let event = pythonize(py, &event).unwrap();

            let py_event_handler = slf.inner.as_ref(py);
            let coro = py_event_handler.call_method(
                "start",
                (event,),
                None,
            ).unwrap(); // this unwrap panics
/*
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value:
    PyErr { type: <class 'RuntimeError'>, value: RuntimeError('no running event loop'), traceback: None }',
    src/builders.rs:43:12
*/

            pyo3_asyncio::tokio::into_future(coro)
        }).unwrap();

        future.await.unwrap();
    }
}

class Handler:
    async def start(self, event):
        print(event)

rust_lib.register_handler(Handler())

[awestlake87]

What you're doing should be possible. The error you're getting indicates to me that either:

• you're calling your start function from a Rust thread that is not aware of the running Python event loop
• or you're calling start before the Python event loop is running.

I'd be able to give a more detailed explanation of the problem and potential solutions if I can see where and how start is being called.

This library does have examples on creating libraries with pyo3-asyncio in the docs. Additionally, these docs might be useful to look at for context on initializing and managing references to the python event loop:

• PyO3 Asyncio Initialization
• Event Loop References (although this kinda gets into the weeds of the library and might not be necessary for your use-case).

[vicky5124]

The event loop is indeed running on a different thread. The library spawns a new tokio task, with a gateway reader, that parses the messages it receives and dispatches them to the event loop trait.

Looking at Event Loop References, I got the idea of doing let current_loop = pyo3_asyncio::get_running_loop(py)?; and storing current_loop on a new field of Event handler, but encountered an issue, can't use generics on #[pyclass] to define a lifetime for the reference.

error: #[pyclass] cannot have generic parameters: lavasnek_rs
  --> src/builders.rs:21:32
   |
21 | pub struct EventHandler<'a> {
   |                        ^^^^

I could store let loop_ref = PyObject::from(current_loop); instead, and use this to run everything:

Python::with_gil(|py| {
    let current_loop = slf.current_loop.cast_as(py).unwrap();
    pyo3_asyncio::tokio::future_into_py_with_loop(current_loop, async move {
        ....
    })
    .unwrap();
});

And it worked! thanks for the information!

[ShadowJonathan]

@vicky5124 i think the "cant have generic parameters" error is a bug, or at least a misappropriated error, maybe its good to file that with the parent pyo3 project?

[awestlake87]

@ShadowJonathan I think in this case it's because the @vicky5124 wants to store the event loop reference inside the #[pyclass]. This is the intended behavior as Python classes defined in Rust must be concrete not generic (even across lifetime parameters).

The solution to this is to use a PyObject like @vicky5124 discovered. The only change I might make to that snippet is using as_ref instead of cast_as to avoid unwrapping the result:

Python::with_gil(|py| {
    let current_loop = slf.current_loop.as_ref(py);
    pyo3_asyncio::tokio::future_into_py_with_loop(current_loop, async move {
        ....
    })
    .unwrap();
});