ROS2 - realsense node becomes unresponsive after it fails to process a request

[akupferb]

---

Info
Camera Model	D435
Firmware Version	5.13.0.50
Operating System & Version	Linux (Ubuntu 20)
Kernel Version (Linux Only)	5.15.0-101-generic
Platform	PC (also occurs on RPi)
Librealsense SDK Version	v2.51.1
Language	C++
Segment	Robot
ROS Distro	Galactic
RealSense ROS Wrapper Version	4.51.1

Issue Description

During runtime, part of the processing of my system requires that the resolution of the color camera stream be changed. In addition, the align_depth parameter is enabled, so that once I've changed the color camera stream, I then enable the depth stream (with pointcloud enabled) so that both streams are aligned at the same resolution.

The realsense2 node requires a resolution change request to also be accompanied by a reset of the stream (off-on) to complete the change. I accomplish this by sending all three "commands" {disable, change resolution, enable} in a single Service call to 'set_parameters'

I have discovered two things from this process: 1) Although the ROS Service server on the realsense2 node sends back a "successful" result to my Client for the 'set_parameters', the processing of resetting the color stream has not been completed, and the timing is such that generally the re-enable of the stream occurs in some milliseconds amount of time after the result is sent back. This is somewhat expected as it is Service request and not an Action goal, so there's some race condition in the timing of the completion. 2) However, since I then intend to enable the depth stream at the same resolution as the color stream after change (for alignment), and I also am making this request using the same 'set_parameters' Service, I find that I have to create a manual delay between when getting the result from the first Service call to sending the next one (to fully allow time for the realsense2 node to finish processing the resolution change and re-enable stream from the first call), as if I don't, I get the following error:

If the second Service call (to enable_depth) is sent while the realsense2 node is still processing the first request (i.e., the color stream was disabled and resolution changed, but not yet re-enabled), it seems the node gets stuck in some handling error between trying to finish processing the previous call and handling the new call, and then the node completely freezes from an output and communication perspective... the current Service request never gets a response (not even a failure), neither stream gets enabled, and any subsequent attempts to communicate with the node fail (service call, param get/set from cmd line, etc) as the node does not respond to anything until it is completely shutdown and restarted. Notably, I have also started to see a couple of times (on a RPi running the same software) this node "freeze" occur even on the first call (to set the color stream), where there isn't a previous call still being processed, and so cannot identify what the cause is.

I am looking for some clarification perhaps of what is happening behind the scenes here, and if there is any dependable solution to prevent this error from occurring...

Thanks.

[MartyG-RealSense]

Hi @akupferb Firstly, can you confirm whether stability improves if you do not enable alignment, please?

In regard to Raspberry Pi, it is typical for there to be problems with RealSense SDK or RealSense ROS wrapper activities when doing more with the Pi than just streaming depth and color individually without alignment or pointcloud.

[akupferb]

Hi MartyG,

I tested the system again with each of the following set to false/disabled, to try and reduce the load: align_depth.enable, pointcloud.ordered_pc, pointcloud.enable, depth_module.emitter_enabled. I get the same "node freeze" results when I do not add a sufficient delay between the call for the color stream and the call for the depth stream... As the issue mainly seems to be related to the processing of the color stream not being complete when the second call comes in, changing the parameters of the depth stream likely had no effect.

For reference, here is my initial config for my camera:

align_depth.enable: True
pointcloud.ordered_pc: True
pointcloud.allow_no_texture_points: True
pointcloud.stream_filter: 0
pointcloud.enable: True
enable_color: False
enable_depth: False
enable_infra1: False
enable_infra2: False
depth_module.profile: '424x240x30'
depth_module.emitter_enabled: 1
depth_fps: 30.0
clip_distance: 6.0
filters: 'decimation,disparity,spatial,temporal'

I can further test with alignment disabled on the Rpi to see if the failure occurrence on the first call still occurs, however infrequently, though am not expecting different results.

I was concerned that this was related to the general instability issues of the RealSense SDK or RealSense ROS wrapper that I have noticed... if this is the case, is there any generalized fix or version change that would help me?

Thanks, Arik

[MartyG-RealSense]

There is little information available about making service calls in the RealSense ROS2 wrapper, unfortunately. And the only service call the wrapper supports is to retrieve information about the camera.

https://github.com/IntelRealSense/realsense-ros?tab=readme-ov-file#available-services

At https://github.com/IntelRealSense/librealsense/issues/12685#issuecomment-1952671164 I suggested instead enabling and disabling streams during runtime with the ros2 param set command. For example:

ros2 param set /camera/camera enable_depth false
ros2 param set /camera/camera enable_depth true

[akupferb]

The service call I refer to is

ros2 service call /camera/camera/set_parameters rcl_interfaces/srv/SetParameters ...

which is the equivalent of the common ones you mention, ros2 param set ... which are command line methods and not able to be sent during runtime from within my nodes...

[MartyG-RealSense]

There are not any references available regarding using a ROS2 service call other than the device information one in the RealSense ROS wrapper documentation so I do not have advice that I can provide about that question, unfortunately. I do apologize.

[MartyG-RealSense]

Hi @akupferb Do you require further assistance with this case, please? Thanks!

[akupferb]

I had to find a workaround since it seems there is no further support in this specific area, as you’ve noted. Unless/until the stability and functionality of the Realsense ROS wrapper is improved, then you can consider this case closed. Thank you for your assistance.

[MartyG-RealSense]

You are very welcome. My apologies that the wrapper did not meet your requirements in this particular aspect.