How to forget memory with the jemalloc strategy.

[fishiu]

Hello, I was using PyO3 to bind this rust code to python and is now struggling with the memory ownership problem (more specifically, double free).

When I try to pass an lmr to python, I do std::mem::forget(lmr) to make rust forget about the memory, as the memory is already somehow transfered to python by some unsafe pointer operations.

However, according to my reasoning, I think the jemalloc strategy does not give up this memory even lmr.drop() is not called. Is this expected? Is there way to forget this memory in jemalloc strategy?

P.S. raw strategy works fine and no double free occurs.

[fishiu]

This is my code

fn rdma_get_item<'a>(&mut self, py: Python<'a>) -> PyResult<&'a PyArray<u8, Dim<[usize; 1]>>> {
    let mut lmr = some_function();
    let lmr_len = (&lmr).length();
    let lmr_data_ptr = *lmr.as_mut_ptr();
    let lmr_data = unsafe { Vec::from_raw_parts(lmr_data_ptr, lmr_len, lmr_len) };

    std::mem::forget(lmr);

    let py_array = lmr_data.into_pyarray(py);
    Ok(py_array)
}

[GTwhy]

jemalloc strategy does not give up this memory even lmr.drop() is not called. I don’t quite understand the meaning of this sentence. Do you mean: Je free the memory even drop is not called?

I think lmr.drop() -> jemalloc::free() will not be called if you use forget(lmr), and the memory should keep the allocated state(is this what you mean give up?).

But, we have not conducted any analysis or testing on transferring MR between different frameworks. Welcome contributions of complete application cases and discussions on potential issues.