search

eclipse-zenoh / zenoh-plugin-ros2dds

A Zenoh plug-in for ROS2 with a DDS RMW.
https://zenoh.io
Other
111 stars 26 forks source link

[Bug] Denying/allowing topics on the subscriber side does not work as expected #241

Open lukaszanger opened 1 week ago

lukaszanger commented 1 week ago

Describe the bug

Denying and allowing topics via the configuration file does not work as expected for zenoh-bridge-ros2dds. In the minimal example setup below, even if explicitely denying a specific topic, it is nevertheless subscribed by the Zenoh bridge (and published as a ROS2 topic).

To reproduce

To reproduce the bug, start the following minimal example via docker compose up.

In the setup, we have a publishing ROS2 node (with a publishing Zenoh bridge) with ROS_DOMAIN_ID=1 in a specific Docker network rosnet-pub that publishes a topic with 1 Hz. On the other hand, we have two subscribing Zenoh bridges both with ROS_DOMAIN_ID=2 and both in Docker network rosnet-sub.

One of the subscribing Zenoh bridges allows the topic /pub. The other subscribing Zenoh bridge denies this topic. Nevertheless, it seems that both of these subscribing Zenoh bridges are publishing the ROS2 topic /pub which is not expected. What is observed: ros2 topic info /pub shows that it is published by two ROS2 nodes. ros2 topic hz /pub gives a frequency in the range of 2000 - 4000 Hz (and not 1 Hz).

Expected outcome: Zenoh subscriber sub1 publishes the ROS2 topic /pub. Zenoh subscriber sub2 does not publish anything. Actual outcome: Both Zenoh subscribers publish the ROS2 topic /pub.

Docker Compose File

docker-compose.yml:

services:

  zenoh-router:
    image: eclipse/zenoh:0.11.0
    restart: unless-stopped
    ports:
      - 7447:7447
    networks:
      - rosnet-pub
      - rosnet-sub

  pub:
    image: rwthika/ros2:latest
    environment:
      - ROS_DOMAIN_ID=1
      - RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
    command:
      - /bin/bash
      - -c
      - |
        apt-get update
        apt-get install -y ros-humble-rmw-cyclonedds-cpp
        ros2 topic pub /pub std_msgs/msg/String "data: 'Topic from pub'" --node-name pub
    networks: 
      - rosnet-pub

  zenoh-bridge-dds-pub:
    image: eclipse/zenoh-bridge-ros2dds:0.11.0
    restart: unless-stopped
    command: -c /config.json5 -d 1
    environment:
      - ROS_DISTRO=humble
    networks: 
      - rosnet-pub
    volumes:
      - ./zenoh_pub.json5:/config.json5

  sub1:
    image: rwthika/ros2:latest
    environment:
      - ROS_DOMAIN_ID=2
      - RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
    command:
      - /bin/bash
      - -c
      - |
        apt-get update
        apt-get install -y ros-humble-rmw-cyclonedds-cpp
        sleep infinity
    networks: 
      - rosnet-sub

  zenoh-bridge-dds-sub1:
    image: eclipse/zenoh-bridge-ros2dds:0.11.0
    restart: unless-stopped
    command: -c /config.json5 -d 2
    environment:
      - ROS_DISTRO=humble
    networks: 
      - rosnet-sub
    volumes:
      - ./zenoh_sub1.json5:/config.json5

  sub2:
    image: rwthika/ros2:latest
    environment:
      - ROS_DOMAIN_ID=2
      - RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
    command:
      - /bin/bash
      - -c
      - |
        apt-get update
        apt-get install -y ros-humble-rmw-cyclonedds-cpp
        sleep infinity
    networks:
      - rosnet-sub

  zenoh-bridge-dds-sub2:
    image: eclipse/zenoh-bridge-ros2dds:0.11.0
    restart: unless-stopped
    command: -c /config.json5 -d 2
    environment:
      - ROS_DISTRO=humble
    networks: 
      - rosnet-sub
    volumes:
      - ./zenoh_sub2.json5:/config.json5

networks:
  rosnet-pub:
  rosnet-sub:

Zenoh Config of publisher

zenoh_pub.json5:

{
  plugins: {
    ros2dds: {
      id: "zenoh_pub",
      nodename: "zenoh_bridge_ros2dds_pub",
      allow: {
        publishers: ["/pub"],
      },
    },
  },
  mode: "client",
  connect: {
    endpoints: [
      "tcp/zenoh-router:7447"
    ]
  },
}

Zenoh Config of subscriber no. 1:

zenoh_sub1.json5:

{
  plugins: {
    ros2dds: {
      id: "zenoh_sub1",
      nodename: "zenoh_bridge_ros2dds_sub1",
      allow: {
        subscribers: ["/pub"],
      },
    },
  },
  mode: "client",
  connect: {
    endpoints: [
      "tcp/zenoh-router:7447"
    ]
  },
}

Zenoh Config of subscriber no. 2

zenoh_sub2.json5:

{
  plugins: {
    ros2dds: {
      id: "zenoh_sub2",
      nodename: "zenoh_bridge_ros2dds_sub2",
      deny: {
        subscribers: ["/pub"],
      },
    },
  },
  mode: "client",
  connect: {
    endpoints: [
      "tcp/zenoh-router:7447"
    ]
  },
}

System info

lreiher commented 1 week ago

Something to add to my colleague's issue description: you may wonder why we are not simply using a single Zenoh bridge on the sub side. The reason is that we are employing Zenoh in a system where we want to dynamically launch and shutdown certain communication channels, so we're effectively launching one bridge per topic.

TomGrimwood commented 6 days ago

Initially I thought that because you do not deny sub2 to publish "/pub", it will be republishing sub1_bridge "/pub", but this did not help, and pub_bridge shouldn't subscribe to it anyway creating a loop.

However, wouldn't having two Zenoh bridges on the same ROS_DOMAIN_ID be incorrect usage of the bridge?

The readme states:

It's important to make sure that NO DDS communication can occur between 2 hosts that are bridged by zenoh-bridge-ros2dds. Otherwise, some duplicate or looping traffic can occur.

Running zenoh-bridge-dds-sub1 and zenoh-bridge-dds-sub2 on ROS_DOMAIN_ID=2 seems to break the system, as expected?

lreiher commented 6 days ago

Thank you for pointing out that sentence from the README. This might be an unfortunate limitation of the zenoh-bridge-ros2dds for us. Using separate ROS_DOMAIN_IDs for both subscribers does not make sense in our use case, as we want them to both operate on the same ROS domain.

Effectively, what we are trying to do is build a dynamic system. Let's say we have two hosts and are already bridging /topic/a between them. At some point in time we want to automatically launch another communication channel for bridging /topic/b and /topic/c. After some more time, we might want to close down the communication channels for /topic/b and /topic/c then.

We come from using mqtt_client for this. We would simply launch another two mqtt_clients (equivalent to zenoh-bridge-ros2dds) on either host. We now wanted to explore the system's performance using Zenoh. However, it looks like we cannot have two zenoh-bridge-ros2dds operating on the same host on the same ROS_DOMAIN_ID.

Another option could be to reconfigure an existing bridge, e.g., via ROS 2 services. I have found https://github.com/eclipse-zenoh/zenoh-plugin-ros2dds/issues/45, but this doesn't really help us directly, I guess. Shutting down the existing bridge and launching a new one is not really an option, because we would lose traffic in the transition.

Any help is appreciated. :)

TomGrimwood commented 5 days ago

From what I understand, if you were able to whitelist topics at runtime, this would be a satisfactory solution?

A very weird idea to try this would be if you are able to have strange topic names.

pub1 publisher whitelist: [".*/pub1/.*"]
sub1 subscriber whitelist [".*/sub1/.*"]
sub2 subscriber whitelist [".*/sub2/.*"]

Then if you create a topic called "/pub1/sub1/battery_state" it would be sent from p1 to s1. And later a topic called "/pub1/sub2/battery_state" it would be sent from p1 to s2. or even "/pub1/sub1/sub2/battery_state" to be sent to both.

Maybe this is not a solution for you but an interesting thought.

This would still require all bridges running on different DOMAIN_ID's.

lreiher commented 5 days ago

Not really sure I understand your idea. However, we definitely need to be on the same ROS_DOMAIN_ID everywhere, as Sub1 and Sub2 (and others) also need to directly talk to each other, without Zenoh.

As I mentioned, we could achieve the same if we had the option to dynamically reconfigure allow/deny of each Zenoh bridge.