Composable node randomly not loading when shared memory is used.

[ksuszka]

Bug report

Required Info:

• Operating System:
  • Shown using official ros:humble docker image
• Installation type:
  • Official ros:humble docker image
• Version or commit hash:
  • ros-humble-cyclonedds/jammy,now 0.10.3-1jammy.20230822.172333 amd64 [installed,automatic]
  • ros-humble-rmw-cyclonedds-cpp/jammy,now 1.3.4-1jammy.20230919.205940 amd64 [installed]
  • ros-humble-iceoryx-binding-c/jammy,now 2.0.3-1jammy.20230623.023950 amd64 [installed,automatic]
  • ros-humble-iceoryx-hoofs/jammy,now 2.0.3-1jammy.20230623.020506 amd64 [installed,automatic]
  • ros-humble-iceoryx-posh/jammy,now 2.0.3-1jammy.20230623.021542 amd64 [installed,automatic]
• DDS implementation:
  • rmw_cyclonedds_cpp
• Client library (if applicable):
  • rclcpp

Steps to reproduce issue

I prepared separate repo with very simple application and launch file to reproduce the issue https://github.com/ksuszka/cyclonedds_iceoryx_components_issue_demo.

Using this repo:

Build docker image:

docker build -f Dockerfile -t cyclone-component-test .

Open three terminal windows.

In the first terminal window start a docker container with the iox-roudi memory server:

docker run -it --shm-size 1GB --rm --name cyclone-component-test cyclone-component-test iox-roudi

Showing correct behavior with SHM disabled

First, run an example without shared memory to see the correct result.

In the second terminal window run the launch file which spawns 90 component_containers and loads a composable node into each:

docker exec -it --env=CYCLONEDDS_URI='<Domain id="any"><SharedMemory><Enable>false</></></>' cyclone-component-test bash -c ". /ws/install/setup.bash && ros2 launch /ws/src/composition.launch.py"

In the third terminal check how many component_containers were created:

docker exec -it cyclone-component-test bash -c "pgrep -f -c component_container"

You should get the number 90.

And now check how many nodes were created:

docker exec -it cyclone-component-test bash -c ". /ws/install/setup.bash && ros2 node list -c"

You should get the number 181 = 1 launch file + 90 containers + 90 composable nodes.

Showing incorrect behavior with SHM enabled

Now abort an execution in the second terminal by Ctrl+C and run the same command enabling shared memory support:

docker exec -it --env=CYCLONEDDS_URI='<Domain id="any"><SharedMemory><Enable>true</></></>' cyclone-component-test bash -c ". /ws/install/setup.bash && ros2 launch /ws/src/composition.launch.py"

In the third terminal check how many component_containers were created:

docker exec -it cyclone-component-test bash -c "pgrep -f -c component_container"

You should again get the number 90.

And now check how many nodes were created:

docker exec -it cyclone-component-test bash -c ". /ws/install/setup.bash && ros2 node list -c"

You will probably get a number lower than 181 - not all composable nodes were loaded.

Expected behavior

Composable nodes are loaded regardless if SHM is enabled/disabled

Actual behavior

Composable nodes sometimes don't load if SHM is enabled.

Additional information

The issue was observed in ROS system with a few dozens of nodes. Starting the system with SHM enabled caused random nodes to not being loaded.

The code repository contains a minimal example which demonstrates the issue using the standard Iceoryx compilation. As far as I tested, the type of composable node loaded doesn't matter, however in that repository the composable node with the /parameter_events publisher disabled is used so a large number of nodes can be spawned without triggering the https://github.com/eclipse-cyclonedds/cyclonedds/issues/1326#issuecomment-1181699596 issue.