eclipse-cyclonedds / cyclonedds

Eclipse Cyclone DDS project
https://projects.eclipse.org/projects/iot.cyclonedds
Other
839 stars 350 forks source link

Is there a way to replicate the deserialization mechanism used in Cyclone DDS, such as the pycdr2 library, but in the C programming language? #2057

Closed HariKarumuri closed 1 month ago

HariKarumuri commented 1 month ago

Issue Description

Background

The project involves integrating a broker between DDS communication to enable cloud clients to receive DDS published messages. The existing implementation has been modified to use an MQTT broker instead of shared memory for message transmission.

Problem

Currently, messages sent to the MQTT broker are encoded (serialized), making it difficult for MQTT clients to interpret the data directly. MQTT clients can subscribe to topics but receive encoded data instead of the expected deserialized messages.

Task

The next step involves implementing deserialization of the data received from DDS before publishing it to MQTT topics. This process is critical as DDS data serialization is a significant portion (30-40%) of the DDS framework.

Request for Assistance

Seeking insights or guidance on implementing the deserialization process effectively to ensure MQTT clients receive readable messages instead of encoded data.

Additional Information

The existing code successfully handles message transmission via MQTT but lacks the deserialization step necessary for MQTT clients to interpret the messages correctly.

Any suggestions or examples related to DDS data deserialization and integration with MQTT would be greatly appreciated.

eboasson commented 1 month ago

I think I understand what you're looking for, but I am not sure how to answer 🙂 Let me try something ...

I understand that today you read serialized data from DDS and publish it in MQTT. That to means you use:

All are fine, I just want to make sure I get the picture right.

The way I understand MQTT, you have to have the data serialized in some form, that is, you can't publish a complex C struct or C++ class like what comes out of the deserializers. So I imagine when you write you want to deserialize the data to publish it in MQTT, you'd like to then re-serialize it in an easy-to-understand form.

In other words, something like transcoding the CDR to JSON. The original CDR (from the early '90s) is nice and simple and transcoding it would be straightforward. The "new" XCDR2 representation that's becoming the norm in DDS is rather complicated, and I wouldn't try to transcode it. Deserializing it to a C struct and then reserializing it as whatever is much simpler.

The dynsub example doesn't cover all types and is certainly not quite producing well-formed JSON, but that's really just laziness on my part. Perhaps something in that direction would work?

HariKarumuri commented 1 month ago

The dynsub example works fine , Thanks a bunch !!