search

s2e-systems / dust-dds

Rust implementation of the Data Distribution Service (DDS)
Apache License 2.0
84 stars 11 forks source link

[ENTITYID_BUILTIN_PUBLICATIONS_WRITER] Should we choose to continue processing sub-messages even if the sequence numbers are not consecutive? #281

Closed YeahhhhzZ closed 5 months ago

YeahhhhzZ commented 6 months ago

Our project has a scenario of communicating with cyclonedds.

In some cases, the data sent by cyclonedds has sequence_number discontinuities:

Real-Time Publish-Subscribe Wire Protocol
    Magic: RTPS
    Protocol version: 2.1
    vendorId: 01.16 (Unknown)
    guidPrefix: 01103de4972c41380c170d32
    Default port mapping: domainId=200, participantIdx=103, nature=UNICAST_METATRAFFIC
    submessageId: INFO_DST (0x0e)
        Flags: 0x01, Endianness bit
        octetsToNextHeader: 12
        guidPrefix: c0a87a01c8641c0000000000
    submessageId: INFO_TS (0x09)
        Flags: 0x01, Endianness bit
        octetsToNextHeader: 8
        Timestamp: Apr 28, 2024 10:02:01.270296175 UTC
    submessageId: DATA (0x15)
        Flags: 0x05, Data present, Endianness bit
        octetsToNextHeader: 308
        0000 0000 0000 0000 = Extra flags: 0x0000
        Octets to inline QoS: 16
        readerEntityId: ENTITYID_BUILTIN_PUBLICATIONS_READER (0x000003c7)
        writerEntityId: ENTITYID_BUILTIN_PUBLICATIONS_WRITER (0x000003c2)
        writerSeqNumber: 1
        serializedData
            encapsulation kind: PL_CDR_LE (0x0003)
            encapsulation options: 0x0000
            serializedData:
    submessageId: INFO_TS (0x09)
        Flags: 0x01, Endianness bit
        octetsToNextHeader: 8
        Timestamp: Apr 28, 2024 10:02:01.374387094 UTC
    submessageId: DATA (0x15)
        Flags: 0x05, Data present, Endianness bit
        octetsToNextHeader: 316
        0000 0000 0000 0000 = Extra flags: 0x0000
        Octets to inline QoS: 16
        readerEntityId: ENTITYID_BUILTIN_PUBLICATIONS_READER (0x000003c7)
        writerEntityId: ENTITYID_BUILTIN_PUBLICATIONS_WRITER (0x000003c2)
        writerSeqNumber: 4
        serializedData
            encapsulation kind: PL_CDR_LE (0x0003)
            encapsulation options: 0x0000
            serializedData:
                PID_TOPIC_NAME
                    parameterId: PID_TOPIC_NAME (0x0005)
                    parameterLength: 36
                    topic: /aos/component/latency_report
                PID_TYPE_NAME
                    parameterId: PID_TYPE_NAME (0x0007)
                    parameterLength: 40
                    typeName: aos::transport::cyclone::DDSMessage
                PID_LIVELINESS
                    parameterId: PID_LIVELINESS (0x001b)
                    parameterLength: 12
                    PID_LIVELINESS
                PID_RELIABILITY
                    parameterId: PID_RELIABILITY (0x001a)
                    parameterLength: 12
                    Kind: RELIABLE_RELIABILITY_QOS (0x00000002)
                PID_HISTORY
                    parameterId: PID_HISTORY (0x0040)
                    parameterLength: 8
                    Kind: KEEP_LAST_HISTORY_QOS (0x00000000)
                    Depth: 128
                PID_DATA_REPRESENTATION
                    parameterId: PID_DATA_REPRESENTATION (0x0073)
                    parameterLength: 8
                    parameterData: 0200000000000200
                Unknown (0x0075)
                    parameterId: Unknown (0x0075)
                    parameterLength: 100
                    parameterData: 6000000001100040280000002400000014000000f1f66653…
                PID_PROTOCOL_VERSION
                    parameterId: PID_PROTOCOL_VERSION (0x0015)
                    parameterLength: 4
                    Protocol version: 2.1
                PID_VENDOR_ID
                    parameterId: PID_VENDOR_ID (0x0016)
                    parameterLength: 4
                    vendorId: 01.16 (Unknown)
                PID_ENDPOINT_GUID
                    parameterId: PID_ENDPOINT_GUID (0x005a)
                    parameterLength: 16
                    Endpoint GUID: 01103de4 972c4138 0c170d32 00000803
                Unknown (0x800c)
                    parameterId: Unknown (0x800c)
                    parameterLength: 4
                    parameterData: 01000000
                PID_SENTINEL
                    parameterId: PID_SENTINEL (0x0001)
    submessageId: INFO_TS (0x09)
        Flags: 0x01, Endianness bit
        octetsToNextHeader: 8
        Timestamp: Apr 28, 2024 10:02:01.375171175 UTC
    submessageId: DATA (0x15)
        Flags: 0x05, Data present, Endianness bit
        octetsToNextHeader: 304
        0000 0000 0000 0000 = Extra flags: 0x0000
        Octets to inline QoS: 16
        readerEntityId: ENTITYID_BUILTIN_PUBLICATIONS_READER (0x000003c7)
        writerEntityId: ENTITYID_BUILTIN_PUBLICATIONS_WRITER (0x000003c2)
        writerSeqNumber: 5
        serializedData
            encapsulation kind: PL_CDR_LE (0x0003)
            encapsulation options: 0x0000
            serializedData:
                PID_TOPIC_NAME
                    parameterId: PID_TOPIC_NAME (0x0005)
                    parameterLength: 24
                    topic: aos_topic_change
                PID_TYPE_NAME
                    parameterId: PID_TYPE_NAME (0x0007)
                    parameterLength: 40
                    typeName: aos::transport::cyclone::DDSMessage
                PID_LIVELINESS
                    parameterId: PID_LIVELINESS (0x001b)
                    parameterLength: 12
                    PID_LIVELINESS
                PID_RELIABILITY
                    parameterId: PID_RELIABILITY (0x001a)
                    parameterLength: 12
                    Kind: RELIABLE_RELIABILITY_QOS (0x00000002)
                PID_HISTORY
                    parameterId: PID_HISTORY (0x0040)
                    parameterLength: 8
                    Kind: KEEP_LAST_HISTORY_QOS (0x00000000)
                    Depth: 128
                PID_DATA_REPRESENTATION
                    parameterId: PID_DATA_REPRESENTATION (0x0073)
                    parameterLength: 8
                    parameterData: 0200000000000200
                Unknown (0x0075)
                    parameterId: Unknown (0x0075)
                    parameterLength: 100
                    parameterData: 6000000001100040280000002400000014000000f1f66653…
                PID_PROTOCOL_VERSION
                    parameterId: PID_PROTOCOL_VERSION (0x0015)
                    parameterLength: 4
                    Protocol version: 2.1
                PID_VENDOR_ID
                    parameterId: PID_VENDOR_ID (0x0016)
                    parameterLength: 4
                    vendorId: 01.16 (Unknown)
                PID_ENDPOINT_GUID
                    parameterId: PID_ENDPOINT_GUID (0x005a)
                    parameterLength: 16
                    Endpoint GUID: 01103de4 972c4138 0c170d32 00000b03
                Unknown (0x800c)
                    parameterId: Unknown (0x800c)
                    parameterLength: 4
                    parameterData: 01000000
                PID_SENTINEL
                    parameterId: PID_SENTINEL (0x0001)
    submessageId: INFO_TS (0x09)
        Flags: 0x01, Endianness bit
        octetsToNextHeader: 8
        Timestamp: Apr 28, 2024 10:02:01.476425548 UTC
    submessageId: DATA (0x15)
        Flags: 0x05, Data present, Endianness bit
        octetsToNextHeader: 316
        0000 0000 0000 0000 = Extra flags: 0x0000
        Octets to inline QoS: 16
        readerEntityId: ENTITYID_BUILTIN_PUBLICATIONS_READER (0x000003c7)
        writerEntityId: ENTITYID_BUILTIN_PUBLICATIONS_WRITER (0x000003c2)
        writerSeqNumber: 6
        serializedData
            encapsulation kind: PL_CDR_LE (0x0003)
            encapsulation options: 0x0000
            serializedData:
                PID_TOPIC_NAME
                    parameterId: PID_TOPIC_NAME (0x0005)
                    parameterLength: 36
                    topic: /aos/component/heartbeat_report
                PID_TYPE_NAME
                    parameterId: PID_TYPE_NAME (0x0007)
                    parameterLength: 40
                    typeName: aos::transport::cyclone::DDSMessage
                PID_LIVELINESS
                    parameterId: PID_LIVELINESS (0x001b)
                    parameterLength: 12
                    PID_LIVELINESS
                PID_RELIABILITY
                    parameterId: PID_RELIABILITY (0x001a)
                    parameterLength: 12
                    Kind: RELIABLE_RELIABILITY_QOS (0x00000002)
                PID_HISTORY
                    parameterId: PID_HISTORY (0x0040)
                    parameterLength: 8
                    Kind: KEEP_LAST_HISTORY_QOS (0x00000000)
                    Depth: 128
                PID_DATA_REPRESENTATION
                    parameterId: PID_DATA_REPRESENTATION (0x0073)
                    parameterLength: 8
                    parameterData: 0200000000000200
                Unknown (0x0075)
                    parameterId: Unknown (0x0075)
                    parameterLength: 100
                    parameterData: 6000000001100040280000002400000014000000f1f66653…
                PID_PROTOCOL_VERSION
                    parameterId: PID_PROTOCOL_VERSION (0x0015)
                    parameterLength: 4
                    Protocol version: 2.1
                PID_VENDOR_ID
                    parameterId: PID_VENDOR_ID (0x0016)
                    parameterLength: 4
                    vendorId: 01.16 (Unknown)
                PID_ENDPOINT_GUID
                    parameterId: PID_ENDPOINT_GUID (0x005a)
                    parameterLength: 16
                    Endpoint GUID: 01103de4 972c4138 0c170d32 00000d03
                Unknown (0x800c)
                    parameterId: Unknown (0x800c)
                    parameterLength: 4
                    parameterData: 01000000
                PID_SENTINEL
                    parameterId: PID_SENTINEL (0x0001)

So when encountering this situation (serial number changes from 1 to 4), Can we choose to raise an alarm and continue processing these sub-messages?

jrebelo commented 6 months ago

Hi @YeahhhhzZ,

Do you notice any interoperability issue between Dust DDS and Cyclone DDS or is this question based on observing the samples?

If I understand correctly this is data coming from the built-in discovery endpoints. These endpoints have a Reliable QoS setting which means that data is guaranteed to be delivered. That is done by either a Data submessage with the expected sequence number arriving or a Gap submessage indicating that the sequence number is not available anymore, in which case the reader can continue with the following sequence numbers. For this particular case you can not just ignore the sequence numbers and continue since this would break the reliability behavior. If you have a Best Effort reader then that one would just skip over the missing sequence numbers and as a user you could get a SampleLost listener event or notification.

YeahhhhzZ commented 6 months ago

Hi @YeahhhhzZ,

Do you notice any interoperability issue between Dust DDS and Cyclone DDS or is this question based on observing the samples?

If I understand correctly this is data coming from the built-in discovery endpoints. These endpoints have a Reliable QoS setting which means that data is guaranteed to be delivered. That is done by either a Data submessage with the expected sequence number arriving or a Gap submessage indicating that the sequence number is not available anymore, in which case the reader can continue with the following sequence numbers. For this particular case you can not just ignore the sequence numbers and continue since this would break the reliability behavior. If you have a Best Effort reader then that one would just skip over the missing sequence numbers and as a user you could get a SampleLost listener event or notification.

OK， thanks! I will continue to investigate why data is lost on the cyclonedds side :)

jrebelo commented 5 months ago

I will close this issue as it doesn't seem to be a problem with Dust DDS. Don't hesitate to re-open if you find more.