Support for a zero-copy take-modify-publish pipeline

[budrus]

Brief feature description

This is a new feature for iceoryx that allows to take a sample received with a subscriber, modify it and pass it to a publisher for sending.

Detailed information

iceoryx supports true zero-copy communication from a publisher to many subscribers. The subscribers get a const reference/pointer to the received sample. This is good idea as a subscriber typically is not interested in changing the received sample. This is even a bad idea as there might be other subscribers reading the same sample in parallel. But sometimes people have the use case in which one topic type is used in a chain with several nodes where each node receives a sample has to modify the data and send it to the next node. With the current design of iceoryx, each node has to do a copy of the sample for being able to modify and forward it. It would be cool to support a take-modify-publish functionality which allows to modify and forward a received sample in a zero-copy way. Constraints would be

1. No other subscribers are allowed for this topic
2. The publisher does not store the last chunk what it does by default today
3. The publisher supports the sending of samples that where not previously allocated from this publisher

[elBoberido]

@budrus the biggest challenge will be to have a ref-count of 1 when the chunk is pushed into the queue. If I recall correctly, currently the chunk is passed to the ChunkDistributor first and only then removed from the UsedChunkList. This would lead to a short time window where the subscriber could not take exclusive ownership of the chunk.

[dkroenke]

See this discussion for further information.

[elBoberido]

Just some remarks for the points from the discussion on the developer meetup.

• Possible problem: How to transfer the ownership of a sample to another publisher if the originator has another mempool/memory segment?
  • this is a similar problem to a subscriber not having read access to a memory segment. Depending on how we craft the API, we could e.g. have a Sample<const T>::upgrade method returning an expected<Sample<T>, Error>
• Should history size > 1 be allowed?
  • semantically I think this cannot work but we could have something like a writer cache which delivers the sample once there is a subscriber. So it would be similar to the history but not persistent in the publisher
• Maybe introduce a "locking" mechanism to sample for forwarding
  • I have no idea what this point means. Can someone elaborate a bit?
• Create a new "exclusive" port type to explicitly forbid history to restrict the container of (subscriber)queues to capacity = 1
  • why the restriction to capacity == 1? For a TMP pipeline it shouldn't matter how large the queue is. Do I overlook something?
  • a new port has some pros and cons. The biggest pro is that we do not reach a point where all the options result in combinatorial explosion which is hard to test and the user cannot easily misconfigure the ports

Another question arises from the subscriber of a TMP pipeline. How do we distinguish between accidentally having multiple subscriber to a TMP port and intentionally having them, e.g. at the end of the pipeline. If the TMP port decides this, we somewhat break encapsulation since we always would have to touch the TMP port application when we change the pipeline on the subscriber side. If the decision is taken on the subscriber side we either have a race between a subscriber and a TMP port or we give precedence to the TMP port and unsubscribe all subscriber once a TMP port subscribes to another TMP port.

[budrus]

@elBoberido

• Maybe introduce a "locking" mechanism to sample for forwarding

  • I have no idea what this point means. Can someone elaborate a bit?

Hmm, maybe we meant the check you propose with

Sample::upgrade method returning an expected<Sample, Error>

• Create a new "exclusive" port type to explicitly forbid history to restrict the container of (subscriber)queues to capacity = 1

  • why the restriction to capacity == 1? For a TMP pipeline it shouldn't matter how large the queue is. Do I overlook something?

I think we meant the size of the container with chunk queues on the publisher side. I.e. we ensure that there can only be one subscriber. If a second one appears we would call the error handler