Node Freezes When Enabling Color, Depth, and PointCloud Simultaneously

[BryanBetancur]

Hello

Required Info
Camera Model	D435i
Firmware Version	5.16.0.1
Operating System	Ubuntu 20.04.6 LTS
Platform	NVIDIA Jetson Orin AGX
Jetpack	5.1.1 [L4T 35.3.1]
Kernel Version (Linux Only)	5.10.104-tegra
Librealsense SDK Version	2.56.1
ROS Distro	Humble
RealSense ROS Wrapper Version	4.56.1

Issue description

When attempting to enable the color, depth, and pointcloud sensors simultaneously, the Realsense node becomes unresponsive. The node stops responding and does not complete the enabling process.

Steps to Reproduce

1. Launch the Realsense node: Configure the node with the following parameters initially set to false:

   enable_color: false
   enable_depth: false
   pointcloud.enable: false

   2. Create a Parameter File: Create a file named params.yaml with the following content, replacing {realsense_node_name} with the actual name of the Realsense node:

{realsense_node_name}:
  ros__parameters:
    enable_color: true
    enable_depth: true
    pointcloud.enable: true

3. Load the Parameters:

In a separate terminal, load the parameters by running:

ros2 param load {realsense_node_name} params.yaml

Expected Behavior

The node should successfully enable the color, depth, and pointcloud sensors and start streaming data.

Actual Behavior

The node becomes unresponsive, and only the following line is printed in the output:

Stopping Sensor: RGB Camera

The service call does not complete, and no further logs or responses are generated.

[MartyG-RealSense]

Hi @BryanBetancur Does the node launch successfully if you do not set pointcloud.enable to true in the yaml file, please?

[asalimil]

I do have the same issue with realsense running in Jetson AGX Orin, Ubuntu 20.04, librealsense v2.55.1 and RealSense v2.3.2.

[MartyG-RealSense]

Hi @asalimil Ideally you should be using librealsense 2.50.0 or 2.51.1 with the 2.3.2 ROS1 wrapper, as the ROS1 wrapper is no longer developed (the ROS2 wrapper continues to be updated) and so 2.3.2 has not been updated for recent librealsense versions.

Before considering changing your librealsense version though, I recommend first checking whether an RGBD launch with the rs_rgbd.launch launch file can successfully publish a pointcloud. There is a two step process to doing this.

1. Install support for an RGBD launch using the installation command for the ROS1 version that you are using.

Kinetic sudo apt-get install ros-kinetic-rgbd-launch

Melodic sudo apt-get install ros-melodic-rgbd-launch

Noetic sudo apt-get install ros-noetic-rgbd-launch

2. Launch rs_rgbd.launch

roslaunch realsense2_camera rs_rgbd.launch enable_pointcloud:=true

The launch file should publish the pointcloud to depth_image_proc.

[BryanBetancur]

Hi Marty, With poincloud in false the same issue happens:

  ros__parameters:
    enable_color: true
    enable_depth: true
    pointcloud.enable: false

I identify that the problem is related with enable_color, when I tried any combination with enable_color in true it gets stuck, but when enable_color is in false there is no problem, for example I tried to enable pointcloud and depth at the same time without inconvenient.

[MartyG-RealSense]

Hi @BryanBetancur Are you able to set color to false and instead texture the pointcloud with the infrared stream using the 'pointcloud.stream_filter' parameter in the launch instruction below, please?

ros2 launch realsense2_camera rs_launch.py enable_infra1:=true enable_color:=false pointcloud.enable:=true pointcloud.stream_filter:=1

[BryanBetancur]

Using that configuration the node also get stuck trying to enable color and depth at the same time.

[MartyG-RealSense]

I was suggesting to not enable color and to use the infrared topic to texture the pointcloud instead. Is the color topic a requirement for your project, please?

[BryanBetancur]

Yes, I need color but, independently of it, I think internally the dynamical parameters have some manage problem that would be checked in the future. Thanks for your help.

[MartyG-RealSense]

Does it still get stuck if you try setting different resolutions and FPS speeds for color in the launch instruction? For example:

ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true rgb_camera.color_profile:=848x480x15

[BryanBetancur]

My current color and depth profile already have exactly that configuration.

[MartyG-RealSense]

What happens if you go lower (640x480) or higher (1280x720) in resolution, please?