Closed appcypher closed 1 month ago
I should add that I'm on commit 46f8b60
. Running on macOS.
This is an API bug on our side. We already discussed some solutions but did not find the time to implement them.
loan_uninit
returns a SampleMut<MaybeUninit<T>>
and this version should not provide a send
method, only SampleMut<T>
should, and this you can acquire with
let sample = publisher.loan_uninit()?;
let sample = sample.write_payload(1234); // SampleMut<MaybeUninit<T>> becomes SampleMut<T>
What happens behind the scenes here is that:
let sample = publisher.loan_uninit()?;
1234
is written into the memory positionSampleMut
is going out of scope and the memory is releasedlet sample = publisher.loan_uninit()?;
acquires the same memory again where still the 1234
is written tosample.send()?;
sends technically uninitialized memory to the subscriber (due to our API bug)1234
becomes available again.1234
content again and the game restarts.But it would be nice to have a way to reuse the same loaned memory in a consistent way.
This is something we consider currently when designing request response, but this is not always possible. Just think of the use case where one client sends a request to two servers and receives two responses. But for a many client to one server scenario it could work.
But if you build request response on top of our current publish subscribe architecture you will not be able to reuse the received sample and write additional data into it. The reason is that every publisher comes with its own data segment and there are internal checks that should ensure that you do not send samples that do not belong to your data segment.
However, happy news, we are aiming to finish the request-response implementation in September/October!
@appcypher Could you tell us a bit more about your use case? Why do you want to reuse the sample/memory and do not want to acquire a new one?
@appcypher it is not possible to reuse the same memory. There is currently a bug on main which allows to send uninitialized samples. it should actually not be possible to have a loan_uninit
followed by a send
. The write_payload
returns the actual sample to be published.
So your code releases the sample immediately after write_payload
. The first iteration of the loop gets that release sample and since the memory is not touched, it holds the value you wrote before. The second iteration gets a new sample since the first one seems to be still in use. To be more precise, it will be in the subscriber queue. On the third iteration, the initial sample is again released and the loan_uninit
gets that sample. The cycle repeats and that's how you get that output.
... oh, your code needs to be changed to this
let publisher = service.publisher_builder().max_loaned_samples(1).create()?;
while let NodeEvent::Tick = node.wait(CYCLE_TIME) {
let sample = publisher.loan_uninit()?;
let sample = sample.write_payload(1234);
sample.send()?;
}
So your code releases the sample immediately after write_payload. The first iteration of the loop gets that release sample and since the memory is not touched, it holds the value you wrote before. The second iteration gets a new sample since the first one seems to be still in use. To be more precise, it will be in the subscriber queue. On the third iteration, the initial sample is again released and the loan_uninit gets that sample. The cycle repeats and that's how you get that output.
Thanks for the explanation.
@appcypher Could you tell us a bit more about your use case? Why do you want to reuse the sample/memory and do not want to acquire a new one?
I thought it would make certain patterns possible. In my case, in trying to implement a contrived request-response communication pattern, I landed on a queue-based solution (there's just one client and one server communicating). The client would write its chunked request to the same in-memory queue (pushing chunks continuously) and the receiver/server would pick it up from there -- sending back some ack. This requires that a publisher holds onto the same memory where it mutates the queue repeatedly. Maybe a kind of sample that can be resent without being owned.
I may be totally off here but I believe no real send
ing is taking place since the pub-sub processes have access to the same memory and it is zero-copy. So send
is really just a signalling mechanism.
However, happy news, we are aiming to finish the request-response implementation in September/October!
Sweet. Can't wait since that is really what I need. I'm willing to build a sub-optimal hack that I can use now until then.
I thought it would make certain patterns possible. In my case, in trying to implement a contrived request-response communication pattern, I landed on a queue-based solution (there's just one client and one server communicating). The client would write its chunked request to the same in-memory queue (pushing chunks continuously) and the receiver/server would pick it up from there -- sending back some ack. This requires that a publisher holds onto the same memory where it mutates the queue repeatedly. Maybe a kind of sample that can be resent without being owned.
I may be totally off here but I believe no real
send
ing is taking place since the pub-sub processes have access to the same memory and it is zero-copy. Sosend
is really just a signalling mechanism.
Well, this is exactly like iceoryx works. The loan
call takes memory shared between the processes and send
adds a pointer to that memory to the queue for the consumer + some bookkeeping, memory synchronization, etc. So you have to loan memory, write data to that memory and then send it. There is also the event messaging pattern which can be combined with publish-subscribe to notify the consumer about new data in the queue, instead of having the consumer poll the queue.
Thanks again @elBoberido!
How would you suggest one create a request-reponse model today using existing messaging patterns? I want to keep using iceoryx2 as is until the request-response feature lands.
Thanks for helping the other day. After running some code today I have a better grasp of how iceoryx2 works. I think I know how I can leverage the internal queue system here. I will be closing this issue as the question was really about understanding how to make iceoryx2 work for my use case.
Although, as suggested, it probably makes sense to prevent send on SampleMut<Service, MaybeUninit<T>, ...>
(Code) Example Of Cumbersome API
Great project you have here. I'm new to iceoryx and shmem IPC in general but I'm trying to model a simple request-reply mechanism on top of the existing pub-sub communication. I understand it is part of the roadmap but just to have something working in the meantime, I made a simple solution that requires reusing the allocated memory for a sample.
So I tried reusing the same memory segment:
I expected the subscriber process to output
Recieved: 1234
repeatedly but instead got:Full code: https://github.com/appcypher/iceoryx2-ipc-test/blob/main/bin/test_pub_loan.rs
Improvement Suggestion
Tbh, I'm not sure if this is the bug or an expected behavior as I haven't dived too deep into the code. But it would be nice to have a way to reuse the same loaned memory in a consistent way.