ROS 2 command line introspection tools don't work when AllowMulticast is set to false

schornakj commented 2 years ago

Bug report

Required Info:

Operating System:
- Ubuntu 20.04
Installation type:
- Binaries
Version or commit hash:
- ROS Rolling from the 11 January 2022 sync
DDS implementation:
- CycloneDDS
Client library (if applicable):
- Both rclcpp and rclpy

Steps to reproduce issue

My CycloneDDS XML file is at ~/.ros/cyclonedds.xml (and I do have the CYCLONEDDS_URI variable set to this path). The XML file contains this:

<?xml version="1.0" ?>
<CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
  <Domain id="any">
    <General>
      <NetworkInterfaceAddress>enp0s31f6</NetworkInterfaceAddress>
      <AllowMulticast>false</AllowMulticast>
    </General>
    <Discovery>
      <ParticipantIndex>auto</ParticipantIndex>
        <Peers>
          <Peer Address="localhost"/>
        </Peers>
      <MaxAutoParticipantIndex>50</MaxAutoParticipantIndex>
    </Discovery>
  </Domain>
</CycloneDDS>

In one terminal window, I run a talker node:

ros2 run demo_nodes_cpp talker

In a second terminal window, I run a listener node:

ros2 run demo_nodes_cpp listener

In a third terminal window, I run command line introspection tools:

ros2 node list
ros2 topic echo /chatter

Expected behavior

The list of nodes should contain /talker and /listener. Echoing the /chatter topic should print each message.

Actual behavior

The list of nodes is empty, and echoing the topic results in an error message saying that the topic is not yet published.

The listener node is able to receive messages published on the /chatter topic, since it prints info logs like [INFO] [1646341834.250108183] [listener]: I heard: [Hello World: 1], so communication between the nodes themselves is working.

Additional information

My network interface connects to a wired network. In general, I am curious about if I have correctly configured my CycloneDDS settings for this situation.

Setting AllowMulticast to true does allow the introspection tools to work, but it usually brings down my internet connection. I would like to be able to both access the internet and have my ROS nodes communicate with each other.

edit: Also I wasn't sure if this would be a better fit for the main CycloneDDS repo, so let me know if I should post there instead.

eboasson commented 2 years ago

I have a bit of trouble getting the right version of ROS 2 working on macOS on an M1 so I haven't even got to the point where I could try reproducing your observations. So for now I only I have a copy of observations/suggestions:

Firstly, re the multicast false or true: there is a third option spdp which is auto-selected when the interface is a WiFi interface. That one relies on multicast for discovery, but then does everything else via unicast. There is also the more intricate configuration option of allowing multicast, but then setting the DefaultMulticastAddress to 0.0.0.0 and defining NetworkPartitions to map some topics to multicast (assuming there are some). That spdp might provide a useful work around for you.

Secondly, IIRC there was a time at which the "ROS 2 daemon", started automatically by operations like ros2 node list would use the default RMW implementation. Now, again IIRC, that was changed at some point, but if I am misremembering and considering that the default RMW changed in Humble is Fast-RTPS, it could mean that everything works fine as long as you use only Cyclone, but that there is an issue with discovery data if you include Fast-RTPS in the mix. There is not a completely trivial way of finding out, I would try configuring Cyclone DDS to write a trace with the process id in the name by adding <Tracing><Verbosity>finest</Verbosity><Out>cdds.log.${CYCLONEDDS_PID}</Out></Tracing> to the configuration (hopefully no typos there ...).

Lastly, I would enable exactly that same trace and see what is going on. Now, this is a punishment I don't want to push onto anyone, so in that case you might want to share the log files you're getting ...

clalancette commented 2 years ago

Secondly, IIRC there was a time at which the "ROS 2 daemon", started automatically by operations like ros2 node list would use the default RMW implementation.

This is still the case. What happens is that when you run a command like ros2 topic list, it first checks if a daemon is running. If it is not, it then fires one up using the RMW specified to the ros2 command. In this case, the user didn't ask for one, so it will start up a daemon using the default RMW. Subsequent commands will always connect to that daemon, regardless of what RMW is asked for on the command-line.

If the user specifies an RMW the very first time they run a command (like RMW_IMPLEMENTATION=rmw_cyclonedds_cpp ros2 topic list), then the daemon will get started with the requested RMW. Just like before, subsequent commands will always connect to that daemon, regardless of what RMW is asked for on the command-line.

This isn't the only way to do it (and I'm not even sure this is the most intuitive way). But this is what is currently implemented.

ros2 / rmw_cyclonedds