Open glennpierce opened 1 year ago
Just to update I can run the loop script like
let future = Python::with_gil(|py| -> PyResult<_> {
let asyncio = py.import("asyncio")?;
// calling the py_sleep method like a normal function returns a coroutine
let coroutine = activators.as_ref(py).getattr("main")?.call1((script_id,))?;
let result = asyncio.call_method1("run", (coroutine,))?;
// convert the coroutine into a Rust future
pyo3_asyncio::tokio::into_future(result)
})?;
future.await?;
However, when calling a pyfunction from the python aync main the program hangs or deadlocks
#[pyfunction]
fn set_store(py: Python, script_id: i32, data: String) -> PyResult<&PyAny> {
pyo3_asyncio::tokio::future_into_py(py, async move {
println!("Never gets here");
// Some rust function
Python::with_gil(|py| {
Ok(Python::with_gil(|py| py.None()))
})
})
}
I will have to revert to 0.13 for now as I have no idea why it hangs
Thanks
Sorry for taking so long to reply. The instantiation of the event loop behaves pretty differently after 0.13 since we had to cover more use-cases. The reason why you're getting that RuntimeError: no running event loop
is because pyo3-asyncio no longer stores a global reference to the event loop. In pyo3-asyncio 0.14+ you have to provide the event loop to your tasks somehow.
Have you read through Event Loop References and ContextVars? Those docs should help explain why it changed / how to handle it in your code
That's ok. I thought I had read through that. I must confess after having no luck I separated out my python / rust through ipc / protocol buffers which was probably a cleaner solution anyway for me.
Thanks
Same issue here.
@victorteokw this error is not usually a bug, but more of a problem of making the code aware of the event loop you're using. I'd still recommend reading through Event Loop References and Context Vars. But if you think it's a bug or you're not sure what to do to fix it in your code, feel free to post an example to reproduce it.
Thanks @awestlake87, I cached task locals and the problem is fixed.
Hi @awestlake87 i'm also facing this issue, could you provide some insight in where my understanding is wrong;
I have a rust pyfunction
#[pyfunction]
fn run(py: Python, py_callable: Py<PyAny>) -> PyResult<()> {
pyo3_asyncio::tokio::run(py, async move {
let handle = tokio::spawn(async move {
let future = Python::with_gil(|py| {
let awaitable = py_callable.call1(py, ()).unwrap();
into_future(awaitable.as_ref(py)).unwrap()
});
let future_result = future.await.unwrap();
println!("Got from future: {:?}", future_result);
});
handle.await.unwrap();
Ok(())
}).unwrap();
PyResult::Ok(())
}
#[pymodule]
fn my_module(_: Python<'_>, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(run, m)?)?;
Ok(())
}
And i'm calling it in python like so:
import my_module
async def foo() -> int:
print("running Python")
await asyncio.sleep(1)
print("Done in Python")
return 3
if __name__ == "__main__":
my_module.run(foo)
Leading to the same RuntimeError
"No running event loop"
I'm using: pyo3 = "0.20.0"
pyo3-asyncio = "0.20.0"
tokio = 1.13
Thanks for your help!
I am trying to port my code to pyo3_asyncio 0.15 from 0.13 And it all compiles. However, when I try to call an async main method in my python script I get the error
sys:1: RuntimeWarning: coroutine 'main' was never awaited RuntimeWarning: Enable tracemalloc to get the object allocation traceback RuntimeError: no running event loop
This used to work fine.
I believe it is because
I spawn threads like
let handle = tokio::spawn(async move { job.get_config().set_running(true); job.run().await; job.get_config().set_running(false); job.get_config().set_have_run(true); job.get_config().update_last_tick(); });
and join them all like
join_all(job_futures).await;
In each thread in job.run() calls my script like
async fn run_python_script(script_id: Option, script_name: &str, script: &str, params: serde_json::Value) -> PyResult<()> {
let (activators, user, version) =
Python::with_gil(|py| -> PyResult<(PyObject, String, String)> {
let sys = py.import("sys")?;
let version: String = sys.get("version")?.extract()?;
let locals = [("os", py.import("os")?)].into_py_dict(py);
let globals = PyDict::new(py);
let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
let user: String = py.eval(code, Some(&globals), Some(&locals))?.extract()?;
let activators = PyModule::from_code(py, script, script_name, script_name)?;
Ok((activators.into(), user, version))
})?;
}
I didn't quite understand the porting documentation here. Do I have to initialise each tokio::spawn thread somehow ?
Sorry for using issues I couldn't find a forum or discord to look for help.
Thanks