Advantages of using CDR over another data rep / serialization scheme?

[jgrussjr]

What are the advantages of using "real" idlc generated CDR-serialized types with cyclonedds (and its bindings)? Phrased another way, what would be the disadvantages of packing bytes into a CDR octet sequence and sending over DDS? On my project, due to a few design factors, we are considering using a different data type definition and serialization scheme but still using DDS. In #162 , @eboasson described that a valid method of using a custom serialization method would be sending it as a CDR octet sequence or by defining one's one ddsi_sertopic, ddsi_serdata, etc. We are leaning towards octet sequence. Either solution seems like a hacky way to use DDS, so I am trying to understand what we would lose by doing this. I think the idea of removing the type awareness of DDS could cause issues I don't understand, so I am trying understand what we would be losing.

Some more obvious things I have thought of:

• No features from CDR (obviously)
• Loss of interoperability with other DDS peers that do not use your custom (de)serialization scheme
• Inability for DDS tools to sniff and represent the data types of the messages (like the python cyclonedds's subscribe and typeof commands) or even other vendor's / project's DDS tooling.
• Degradation of cyclonedds's ability to check for type mismatches during discovery. it doesn't know the format, so if its octets and the right topic name, things will match.
• Inability to use content filtering / querying.

Some areas I am concerned about, but of which I do not have a full grasp:

• Effects on performance of internal Queues - WHC, RHC.
• Effects on shared memory. Does the iceoryx integration make optimizations based on having access to the type information? Or are the only optimizations by virtue of a maximum size for bounded types?
• Am I introducing extra copies / allocations somewhere due to this?
• Any other performance degradation
• Are there any upcoming cyclone features that would have degraded performance?
• Are there ways to mitigate these downsides?

Thanks in advance for your consideration!

[eboasson]

I think your grasp of the pros and cons is rather complete — at least I can't think of any non-obvious ones you forgot, certainly not as long as you're ok with using just a single language binding (the C one is probably easiest to work with for this type of thing). The reason is very simple: before you know it, you want to represent things differently in C++ than in C, and that can easily lead to having to do things twice because the C++ API relies on topic traits. These details can be completely ignored if you're willing to call some C functions from C++.

Effects on performance of internal Queues - WHC, RHC.

These operate on serdata objects, and how those are handled internally are irrelevant to them. So none that I can think of.

Effects on shared memory. Does the iceoryx integration make optimizations based on having access to the type information? Or are the only optimizations by virtue of a maximum size for bounded types?

It only cares about whether the type has a bounded size or not. I think the bigger issue is that writing a custom data representation is a bit of unpleasant work, and adding integration with Iceoryx adds more to the to-do list, and more cases to consider.

If you don't mind a bit of elbow grease, then it'll be fine. We'll also be more than happy to answer any questions you might run into. (There are already a number of issues with rather detailed responses on this, so searching the issue list would perhaps already answer many of them before you even get started.)

Am I introducing extra copies / allocations somewhere due to this?

That depends entirely on how you organise the internal storage of the samples. Sometimes allocating & copying is the easiest approach, and often that means it is a good first approach, but there is nothing special about "custom" representations.

Any other performance degradation

None that I can think of.

Are there any upcoming cyclone features that would have degraded performance?

No. What I do expect to happen is that there'll be some changes in this "serdata" interface in the future. Nothing that I'd be worried about — actually, the interface can only improve ... — and once again we'd be happy to provide guidance on how to deal with those changes.

Are there ways to mitigate these downsides?

I don't think so.