Questions about receive threads

[hqrasbora]

1- When the AllowMulticast option is set to true, all of the receive threads (recv, recvMC, and recvUC) are used. Could you elaborate on the specific purposes of these threads and provide clarification regarding the types of data they are designed to handle?

2- When the AllowMulticast option is enabled, I've observed that the recvMC thread is activated even when data from non-subscribed topics is received. Although this doesn't significantly impact the execution time of the recvMC thread, I'm curious if there is a method to prevent or mitigate these invocations. Additionally, it seems that the recvUC thread is only triggered for the reception of subscribed topics.

[eboasson]

The model Cyclone uses is that it has a small number of sockets created at initialization, and then one additional socket per participant if ManySocketsMode is set. This latter isn't set by default, and it apparently isn't needed because there appear to be no remaining DDS implementations that assume the unicast locator uniquely addresses a single participant because that assumption in the end creates much more trouble than it is worth. Still, the option exists.

The spec uses separate port numbers for discovery data and for application data. There is no difference whatsoever between the two, other than that the spec says a different port number is to be used. It does allow me to refer to data as "discovery" vs "application" below, but Cyclone never takes the port number into consideration when interpreting data.

All sockets can be handled by a single thread recv, and this is the case if MultipleReceiveThreads is false (or by default on Windows because an independently problem was misdiagnosed and we didn't bother to flip the switch back).

If multiple threads are enabled, then, if among the sockets created at initialization there is one dedicated to unicast application data, that socket is handled by recvUC. Similarly, if multicast is supported and enabled for application data, then the socket used for multicast application data is handled by recvMC. Handling it on a separate thread reduces the overhead a bit and allows for a bit more parallelism. It has no functional consequences.

All discovery data, as well as the participant-specific sockets are always handled by the generic recv one. It was done based on the idea that this would give a reasonable balance between number of threads and likelihood that low latency actually matters. Also it keeps us honest in not accidentally forgetting that there may be multiple threads 🙂

The trouble with multicast in general is that anyone subscribed to the multicast group will get the data, and so joining the default multicast group does indeed bring the risk of receiving data you're not subscribed to. The only way to avoid that is to make sure you don't join those multicast groups. The default settings are not good for that ... then again, they came of age in a time and place where receiving a significant amount of multicast data that was of no interest was not an issue. Either because there was little such data or because the load wasn't an issue.

The protocol is built around the readers advertising the addresses on which they want to receive the data, with a fallback to whatever was advertised in the participant if you don't specify anything in the reader discovery data. In Cyclone:

• The SPDPMulticastAddress controls the multicast address used for participant discovery. This is the only address spec'd (default is 239.255.0.1).
• The DefaultMulticastAddress controls the default address for data in the participant and defaults to the participant discovery address if multicast is allowed. You can set it to 0.0.0.0 if you don't want to advertise by default a multicast address for receiving data. Ideally this would be different from the participant discovery one because everyone has to subscribe to that one and IP multicast doesn't take the UDP port number into consideration, but there is no reserved "data multicast address" ...
• You can furthermore define NetworkPartitions (historical term ... perhaps I should rename it one day) where you can map topics by name (with wildcards) to multicast address. If a topic matches one of these, the reader advertises that multicast address.

Untested example of the latter that maps everything in (DCPS) partition "a" to 239.255.0.2 and everything else to 239.255.0.3:

<CycloneDDS xmlns="https://cdds.io/config">
  <Domain Id="any">
    <General>
      <Interfaces>
        <NetworkInterface name="en0" priority="default"/>
      </Interfaces>
    </General>
    <Partitioning>
      <NetworkPartitions>
        <NetworkPartition Name="a" Address="239.255.0.2"/>
        <NetworkPartition Name="b" Address="239.255.0.3"/>
      </NetworkPartitions>
      <PartitionMappings>
        <PartitionMapping DCPSPartitionTopic="a.*" NetworkPartition="a"/>
        <PartitionMapping DCPSPartitionTopic="*.*" NetworkPartition="b"/>
      </PartitionMappings>
    </Partitioning>
  </Domain>
</CycloneDDS>

In other words, whatever the recvUC thread receives really was sent to that process, and only because it actually needs that data. What the recvMC thread receives is what some process in the network deemed necessary to send to a multicast address the process subscribed to. Usually that means there are several readers for that data, but not necessarily on your machine — and because of the way the IP multicast/UDP port split works out, it may even be data for another domain.

The easiest way to eliminate accidental traffic is to use your own unique multicast addresses for everything (or everything other than participant discovery). Or to simply avoid multicast altogether, for that there is an easy way: set AllowMulticast to spdp and it will only use it for participant discovery, nothing else. (This is the default if the network interface is a WiFi interface, by the way.)