awestlake87 / pyo3-asyncio

Other
302 stars 46 forks source link

Question: local_future_into_py() with self and lifetimes #40

Closed pypt closed 3 years ago

pypt commented 3 years ago

Hello! Thank you for a great module, and sorry for a noob question.

For my Python library, I'm wrapping a field of type Box <dyn Core> into a struct WrappedCore as such:

#[pyclass(name = "Core")]
struct WrappedCore {
    pub(crate) internal: Box<dyn Core>,
}

WrappedCore is supposed to be a Python-exposed wrapper around an underlying trait Core which is defined as:

#[async_trait::async_trait]
pub trait Core: Send + Sync {

    async fn register_worker(&self, config: WorkerConfig) -> Result<(), WorkerRegistrationError>;

    // <...>

}

I'm then trying to implement an async proxy method to register_worker():

#[pymethods]
impl WrappedCore {
    fn register_worker<'p>(&mut self, py: Python<'p>, config: WrappedWorkerConfig) -> PyResult<&'p PyAny> {
        pyo3_asyncio::tokio::local_future_into_py(py, async move {
            match self.internal.register_worker(config.internal).await {
                Err(err) => Err(PyOSError::new_err(format!(
                    "{}",
                    err.to_string()
                ))),
                Ok(()) => {
                    Python::with_gil(|py| Ok(py.None()))
                }
            }
        })
    }
}

but it doesn't compile because:

error[E0759]: `self` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
  --> src/lib.rs:72:66
   |
71 |       fn register_worker<'p>(&mut self, py: Python<'p>, config: WrappedWorkerConfig) -> PyResult<&'p PyAny> {
   |                              --------- this data with an anonymous lifetime `'_`...
72 |           pyo3_asyncio::tokio::local_future_into_py(py, async move {
   |  __________________________________________________________________^
73 | |             match self.internal.register_worker(config.internal).await {
74 | |                 Err(err) => Err(PyOSError::new_err(format!(
75 | |                     "{}",
...  |
81 | |             }
82 | |         })
   | |_________^ ...is captured here...
   |
note: ...and is required to live as long as `'static` here
  --> src/lib.rs:72:9
   |
72 |         pyo3_asyncio::tokio::local_future_into_py(py, async move {
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I've googled out that this is probably due to me trying to use self inside local_future_into_py(). How can I access self.internal from within Tokio future definition?

awestlake87 commented 3 years ago

Would it be possible to change your internal: Box<dyn Core> to an Arc<dyn Core> / Rc<dyn Core> and clone it before it's captured by the async move { }? I noticed your register_worker function only needs &self not &mut self, but if you need mutability maybe a Rc<RefCell<_>> or Arc<Mutex<_>> could help.

pypt commented 3 years ago

Thank you so much!

For the reference of others, I've wrapped internal into an Arc:

#[pyclass(name = "Core")]
struct WrappedCore {
    pub(crate) internal: Arc<dyn Core>,
}

and cloned it before using it:

impl WrappedCore {
    fn register_worker<'p>(&self, py: Python<'p>, config: WrappedWorkerConfig) -> PyResult<&'p PyAny> {

        // Clone "internal" to be able to later use it within local_future_into_py()
        let internal = self.internal.clone();

        pyo3_asyncio::tokio::local_future_into_py(py, async move {
            match internal.register_worker(config.internal).await {
                Err(err) => Err(WorkerRegistrationError::new_err(format!(
                    "{}",
                    err.to_string()
                ))),
                Ok(()) => {
                    Python::with_gil(|py| Ok(py.None()))
                }
            }
        })
    }