search

ros2 / rmw_cyclonedds

ROS 2 RMW layer for Eclipse Cyclone DDS
Apache License 2.0
112 stars 91 forks source link

Gzserver crashes on ROS 2 Foxy w Cyclone DDS #395

Closed QiangWuCN closed 2 years ago

QiangWuCN commented 2 years ago

Bug report

Required Info:

Steps to reproduce issue

RMW_IMPLEMENTATION=rmw_cyclonedds_cpp  gazebo -s libgazebo_ros_init.so /usr/share/gazebo-11/worlds/elevator.world

Expected behavior

Gazebo runs smoothly. Tested successfully w/ Fast DDS.

Actual behavior

Gzserver crashes within several minutes after starting with the following logging.

Warning [Model.cc:216] Non-unique name[door] detected 2 times in XML children of model with name[elevator].
Warning [Model.cc:309] Joint with name [door] in model with name [elevator] has a name collision, changing joint name to [door_joint].
1652881428.257805 []   gzserver: create_thread: gzserver: no free slot

Additional information

Ubuntu 20.04 + ROS 2 Foxy + Gazebo 11 is being used in a container, while the host OS is Ubuntu 20.04 and the Docker version is version 20.10.16.

Tried to enlarge the receive buffer size by using the below command, but didn't resolve the problem:

sudo sysctl -w net.core.rmem_max=8388608 net.core.rmem_default=8388608

eboasson commented 2 years ago

@QiangWuCN I suspect it is the same as https://github.com/eclipse-cyclonedds/cyclonedds/issues/479, so it would be worthwhile to try changing that if it is possible for you to do a custom build of cyclonedds.

QiangWuCN commented 2 years ago

@eboasson Thanks a lot. The problem is fixed by building a custom build of cyclonedds.

As for a 128 cores machine, the max number of threads is changed from "64" to "256" (Tested "128" but still failed) in the source file: cyclonedds/src/core/ddsc/src/dds_init.c Line 113 in b84eee5

 thread_states_init (64);

Just for the record, the following branches of cyclonedds and rmw_cyclonedds are used:

git clone https://github.com/eclipse-cyclonedds/cyclonedds eclipse-cyclonedds/cyclonedds -b release/0.7.x
git clone https://github.com/ros2/rmw_cyclonedds ros2/rmw_cyclonedds -b foxy