eclipse-iceoryx / iceoryx2

Eclipse iceoryx2™ - true zero-copy inter-process-communication in pure Rust
https://iceoryx.io
Apache License 2.0
444 stars 22 forks source link

Implement placement new concept #224

Closed elfenpiff closed 2 weeks ago

elfenpiff commented 1 month ago

Brief feature description

See: #220

When a large data type that does not fit into the stack is transmitted via shared memory we encounter a stack overflow. It is caused by a lacking placement new functionality in Rust, see: https://github.com/rust-lang/rust/issues/53827

Example:

let a = Box::new([-1; 3000000]);

We need to add a mechanism so that

let sample = publisher.loan()?;

and

let sample = publisher.loan_uninit()?;
let sample = sample.write_payload(LargeDataType::default());

perform a placement new into the provide shared memory.

Detailed information

One possible solution can be the introduction of a PlacementNew trait in combination with a placement_new derive macro.

elBoberido commented 1 month ago

There is already a workaround for this available by using

let mut sample = publisher.loan_uninit()?;
let sample = unsafe {
    sample
        .payload_mut()
        .as_mut_ptr()
        .write(LargdDataType::default());

    sample.assume_init()
};

It should work reliably with release builds but might fail with debug builds.

We need a long term solution but I think a build-in Rust solution would be ideal. Maybe this is something we could engage in the Rust community and try to speed it up?

elfenpiff commented 1 month ago

@elBoberido, we could try, but I doubt we can realize this in the following months.

I can't entirely agree with calling this reliable when this algorithm depends on a release optimization. For whatever reason, this could be turned off in some corner cases, or the compiler may not be able to realize that it can perform this optimization.

And what if the user, who is using iceoryx, integrates this into unit tests - then it will cause problems again. With this we would also enforce our users to always build in release mode, also during development, which would cause additional problems.

So my take on this is that we require a quick and comfortable solution and we can use our solution as a discussion basis for the Rust community to come to a build-in Rust solution. What do you think?

elBoberido commented 1 month ago

@elfenpiff I think there are more pressing issues but it's up to you.