Issues with RealSense D455 on Jetson Orin NX 16GB with ROS2 Humble and JetPack 6.0

[denysderihlazov]

Required Info
Camera Model	D455
Firmware Version	5.16.0.1
Operating System & Version	Ubuntu 22.04.4 with JetPack 6.0 rev 2
Kernel Version (Linux Only)	5.15.136-tegra
Platform	NVIDIA Jetson Orin NX 16GB
SDK Version	ROS2 Humble
Language	C++, Python }
Segment	Robotics

Issue Description:

Hello everyone,

I am experiencing issues with the Intel RealSense D455 camera on my Jetson Orin NX 16GB running Ubuntu 22.04.4 with JetPack 6.0 rev 2.

System details:

Jetson Orin NX 16GB with original power supply, running in performance mode.
Intel RealSense D455 with firmware version 5.16.0.1.
Running ROS2 Humble.
RealSense camera works perfectly on a Windows 11 laptop.
Using the original cable from the D455 box.

What works:

The camera is detected using lsusb:

Bus 002 Device 007: ID 8086:0b5c Intel Corp. Intel(R) RealSense(TM) Depth Camera 455

The camera works fine when using the realsense-viewer utility.

Problem:

When I try to use the camera through ROS2 or other tools (e.g., rs-fw-update), it fails to detect the camera properly. The following message appears when I check the system logs:

bash

sudo dmesg | grep realsense [ 145.379196] usb 2-1.4: Process 7642 (realsense-viewe) called USBDEVFS_CLEAR_HALT for active endpoint 0x84 [ 145.531881] usb 2-1.4: Process 7571 (realsense-viewe) called USBDEVFS_CLEAR_HALT for active endpoint 0x82

Steps Taken:

Installed librealsense using:

bash

git clone https://github.com/IntelRealSense/librealsense.git
cd librealsense
mkdir build && cd build
cmake .. -DFORCE_RSUSB_BACKEND=ON -DBUILD_WITH_CUDA=true
make -j4
sudo make install

Checked all USB permissions (using sudo chmod a+rw /dev/video*), and added the user to the video group.

The camera shows up in lsusb and v4l2-ctl --list-devices, but tools like rs-fw-update cannot detect it:

bash

rs-fw-update -l
There are no connected devices

Tested with JetPack 6.0 rev 2 and JetPack 6.1, but the same issue persists.

Request:

Could anyone help me resolve this issue, or point me to the right direction on how to make the D455 work with ROS2 on Jetson Orin NX? It seems like the camera works only in realsense-viewer, but does not interact with ROS or rs-fw-update.

Thanks in advance!

[denysderihlazov]

My goal is to set up RTAB-Map or SLAM as shown in the screenshot:

Here’s the corrected version of your message:

My goal is to set up RTAB-Map or SLAM as shown in the screenshot:

obraz

I've noticed that the realsense-viewer stops working as soon as I install the realsense-ros2 packages. After installing the ROS2 RealSense drivers, my Intel RealSense D455 camera is no longer visible in the realsense-viewer, although it is still detected by lsusb. However, if I uninstall everything related to the rs-ros2 packages using the following command:

sudo apt remove ros-humble-realsense2-camera ros-humble-realsense2-camera-msgs

the camera starts working again in realsense-viewer. I've been struggling with this issue for more than a month, and I really need to make the camera visible in Rviz in order to set up SLAM/RTAB-Map together with my lidar.

When I try to run the camera via realsense-viewer, I see these error messages (on my Windows 11 laptop, the camera works perfectly):

[MartyG-RealSense]

Hi @denysderihlazov Have you tried building the ROS2 wrapper from source code instead of from packages to see if the problem still occurs, please?

Install from source

1. Create a ROS2 workspace

mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src/

2. Clone the latest ROS2 Intel® RealSense™ wrapper from https://github.com/IntelRealSense/realsense-ros.git into ~/ros2_ws/src/

git clone https://github.com/IntelRealSense/realsense-ros.git -b ros2-master
cd ~/ros2_ws

3. Install dependencies

sudo apt-get install python3-rosdep -y
sudo rosdep init # "sudo rosdep init --include-eol-distros" for Foxy and earlier
rosdep update # "sudo rosdep update --include-eol-distros" for Foxy and earlier
rosdep install -i --from-path src --rosdistro $ROS_DISTRO --skip-keys=librealsense2 -y

4. Build

colcon build

5. Source environment

ROS_DISTRO=<YOUR_SYSTEM_ROS_DISTRO>  # set your ROS_DISTRO: iron, humble, foxy
source /opt/ros/$ROS_DISTRO/setup.bash
cd ~/ros2_ws
. install/local_setup.bash

[denysderihlazov]

As you can see on a screenshot, I've done it as you mentioned,

Here is my attempt to open realsense-viewer As you can see it is not working

The same when I try to use ROS2 command. I've found a way to open it via ROS2:

Step 1: Clone the Repository

Navigate to your workspace directory and clone the RealSense repository:

bash:

cd ~/ros2_ws
git clone https://github.com/IntelRealSense/librealsense.git
cd librealsense

Step 2: Build with RSUSB and CUDA Support

Create a build directory, configure CMake with RSUSB and CUDA enabled, and compile the SDK:

bash:

mkdir build && cd build
cmake ../ -DFORCE_RSUSB_BACKEND=ON -DBUILD_WITH_CUDA=ON -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
sudo make install

Step 3: Install udev Rules for RealSense

Copy the necessary udev rules for device permissions and reload them:

bash:

sudo cp ../config/99-realsense-libusb.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && sudo udevadm trigger

Step 4: Verify the Installation

Run the RealSense Viewer to check if the camera is detected:

bash: realsense-viewer

Once the camera is detected correctly, you can proceed with installing the ROS2 driver for RealSense. This is how it works via realsense-viewer:

And it also works via Rviz: I've added everything via Add->By topic -> Image

but as you can see with some issues, I can't see cloudpoints2 on Rviz and also depth camera (IR sensor is working and I can see it on a D455 module)

P.S. red dots on Rviz it is Lidar, but nothing related to the Intel Realsense D455 camera.

[MartyG-RealSense]

Thanks so much @denysderihlazov for your detailed feedback and instructions of what worked for you!

Does it make a difference if you add initial_reset:=true to your ros2 launch instruction to reset the camera at launch?

Is depth able to publish in RViz if you do not enable the pointcloud filter in the launch?

Have you closed down realsense-viewer before performing the ROS launch, as the RealSense ROS wrapper does not require realsense-viewer to be running.

[2805651606]

My goal is to set up RTAB-Map or SLAM as shown in the screenshot:

Here’s the corrected version of your message:

My goal is to set up RTAB-Map or SLAM as shown in the screenshot:

obraz

I've noticed that the realsense-viewer stops working as soon as I install the realsense-ros2 packages. After installing the ROS2 RealSense drivers, my Intel RealSense D455 camera is no longer visible in the realsense-viewer, although it is still detected by lsusb. However, if I uninstall everything related to the rs-ros2 packages using the following command:

sudo apt remove ros-humble-realsense2-camera ros-humble-realsense2-camera-msgs

the camera starts working again in realsense-viewer. I've been struggling with this issue for more than a month, and I really need to make the camera visible in Rviz in order to set up SLAM/RTAB-Map together with my lidar.

When I try to run the camera via realsense-viewer, I see these error messages (on my Windows 11 laptop, the camera works perfectly):

你好，请问你是如何正常使用rtab-map的，我在安装rtab-map ros2版本时： git clone https://github.com/introlab/rtabmap.git src/rtabmap git clone --branch ros2 https://github.com/introlab/rtabmap_ros.git src/rtabmap_ros rosdep update && rosdep install --from-paths src --ignore-src -r -y export MAKEFLAGS="-j6" # Can be ignored if you have a lot of RAM (>16GB) colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release

执行到rosdep update && rosdep install --from-paths src --ignore-src -r -y之后，发现在使用realsense-viewer就再也打不开相机了

orin@ubuntu:~/ros2_rt$ realsense-viewer 26/10 12:29:06,526 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.0/video4linux/video0 26/10 12:29:06,732 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.0/video4linux/video1 26/10 12:29:06,733 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.0/video4linux/video2 26/10 12:29:06,734 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.0/video4linux/video3 26/10 12:29:06,735 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video4 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.3/video4linux/video4 26/10 12:29:06,737 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video5 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.3/video4linux/video5 26/10 12:29:07,106 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.0/video4linux/video0 26/10 12:29:07,108 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.0/video4linux/video1 26/10 12:29:07,108 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.0/video4linux/video2 26/10 12:29:07,110 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.0/video4linux/video3 26/10 12:29:07,110 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video4 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.3/video4linux/video4 26/10 12:29:07,111 INFO [281472984776736] (backend-v4l2.cpp:753) Enumerating UVC video5 realpath=/sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.3/2-1.3:1.3/video4linux/video5 26/10 12:29:07,119 INFO [281472984776736] (context.cpp:128) Found 0 RealSense devices (0xff requested & 0xff from device-mask in settings) 你好，请问你是如何解决的

[MartyG-RealSense]

Hello @2805651606 Your question has been replied to at https://github.com/IntelRealSense/librealsense/issues/13467#issuecomment-2441458713 - thank you!

---

您好@2805651606 您的问题已在 https://github.com/IntelRealSense/librealsense/issues/13467#issuecomment-2441458713 回复 - 谢谢！

[MartyG-RealSense]

Hi @denysderihlazov Do you require further assistance with this case, please? Thanks!

[denysderihlazov]

Thanks so much @denysderihlazov for your detailed feedback and instructions of what worked for you!

Does it make a difference if you add initial_reset:=true to your ros2 launch instruction to reset the camera at launch?

Is depth able to publish in RViz if you do not enable the pointcloud filter in the launch?

Have you closed down realsense-viewer before performing the ROS launch, as the RealSense ROS wrapper does not require realsense-viewer to be running.

Hi, thank you for your response. I’ll check it as soon as I resolve this issue. My camera is definitely connected, and there is enough power; the 25W mode (the most powerful one) is enabled. This situation is quite strange to me. Maybe I need to install some additional drivers.

Also, I am experiencing the same issue when I try to run it via realsense-viewer.

Okay, I found my old commands to start the camera. There are still some issues, but I can see the RGB and Depth cameras (though not pointclouds2). Thanks to these commands, I can run the camera after a reboot, which wasn’t possible before I discovered them.

Here are the commands:

sudo modprobe -r uvcvideo
sudo modprobe uvcvideo
 export RS2_USB_BACKEND=ON
 sudo cp ~/ros2_ws/librealsense/config/99-realsense-libusb.rules /etc/udev/rules.d/
 sudo udevadm control --reload-rules && sudo udevadm trigger
 sudo systemctl enable realsense_uvcvideo_reload.service

 export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
 export RMW_FASTRTPS_USE_SHM=0

 echo 'export RMW_IMPLEMENTATION=rmw_fastrtps_cpp' >> ~/.bashrc
 echo 'export RMW_FASTRTPS_USE_SHM=0' >> ~/.bashrc
 source ~/.bashrc

 echo $RS2_USB_BACKEND

Here below result of what you asked me to check Here below I turned on Pointcloud2 via Rviz2 And after that I see errors:

[MartyG-RealSense]

@denysderihlazov I went through your case again carefully from the beginning to analyze it again.

In your latest RViz image, I note that Fixed Frame has a red error stating that map does not exist.

Does anything appear in RViz if the Fixed Frame setting is changed from map to camera_link

[denysderihlazov]

@denysderihlazov I went through your case again carefully from the beginning to analyze it again.

In your latest RViz image, I note that Fixed Frame has a red error stating that map does not exist.

Does anything appear in RViz if the Fixed Frame setting is changed from map to camera_link

Hi:) Sorry for not answering for long time. I'm currently on my last university year and I have lots things to do. This project is my Engineer project for diploma. I have everything done except working Intel Realsense D455 for my RTAB

I have only 1.5 month to make it working properly.

Actually I have tried this:

git clone https://github.com/IntelRealSense/realsense-ros.git -b ros2-master cd ~/ros2_ws

sudo apt-get install python3-rosdep -y
sudo rosdep init # "sudo rosdep init --include-eol-distros" for Foxy and earlier
rosdep update # "sudo rosdep update --include-eol-distros" for Foxy and earlier
rosdep install -i --from-path src --rosdistro $ROS_DISTRO --skip-keys=librealsense2 -y

ROS_DISTRO=humble 
source /opt/ros/$ROS_DISTRO/setup.bash
. install/local_setup.bash

but the result is always the same
denys@ubuntu:~/ros2_ws$ ros2 launch realsense2_camera rs_launch.py
[INFO] [launch]: All log files can be found below /home/denys/.ros/log/2024-11-15-14-19-10-236436-ubuntu-10950
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-1]: process started with pid [10951]
[realsense2_camera_node-1] [INFO] [1731676750.859186212] [camera.camera]: RealSense ROS v4.55.1
[realsense2_camera_node-1] [INFO] [1731676750.859439897] [camera.camera]: Built with LibRealSense v2.55.1
[realsense2_camera_node-1] [INFO] [1731676750.859485053] [camera.camera]: Running with LibRealSense v2.55.1
[realsense2_camera_node-1] [WARN] [1731676750.875672390] [camera.camera]: No RealSense devices were found!
^Z
[1]+  Stopped                 ros2 launch realsense2_camera rs_launch.py
denys@ubuntu:~/ros2_ws$ ^C

I will probably start from the beginning. I have one more SSD NVMe I'm gonna instal Jetpack 6 rev2 (the Jetpack version for Ubuntu 22.04.4)

If you have full list of what to install from like clean OS Ubuntu 22.04.4 to have SLAM/RTAB on Intel Realsense D455 List of commands one by one Thank you in advance:)

[MartyG-RealSense]

An Nvidia guide published recently about using isaac_ros_visual_slam with a RealSense IMU-equipped camera (such as D455) may be a helpful reference.

https://nvidia-isaac-ros.github.io/concepts/visual_slam/cuvslam/tutorial_realsense.html

[denysderihlazov]

Hi MartyG-RealSense,

Thank you for pointing out the Nvidia guide. I have already reviewed it and attempted to follow its steps, but I am still encountering issues. I’ve tried multiple approaches and have been stuck for quite some time now.

Would it be possible for us to schedule a screensharing session to troubleshoot this problem together? I’m happy to compensate you for your time if needed.

If we manage to resolve the issue, I’d be glad to create a detailed tutorial to help others set up the RealSense D455 with Isaac ROS Visual SLAM on Jetson Orin NX, JetPack 6.0+, Ubuntu 22.04.4, and ROS2 Humble.

This is my uni diploma project and I have 1 month to finish it. I have Lidar, motors, power suply (totally everything except Intel Realsense D455 Slam/RTAB).

Please let me know if this works for you.

[MartyG-RealSense]

I am not able to engage in communication activities such as screen-sharing, remote control or live-chat and can only provide advice through comments on this support forum, unfortunately.

---

JetPack 6 has problems with IMU-equipped RealSense cameras because JP6 removed a function called hiddraw that RealSense IMU cameras (which are known as HID devices) make use of. These problems can manifest as no IMU data available or the camera not being detected at all.

Would it be possible for you to change Ubuntu 22.04 to 20.04 and use JetPack 5.1.3 instead of JP6? RealSense IMU cameras should be able to work with JP 5.1.3.

---

If you need to use 22.04 and JP6 then IMU support for JP6 on a USB cable connection can be added by installing the RealSense MIPI driver at the link below, though it is a complex installation process.

https://github.com/IntelRealSense/realsense_mipi_platform_driver

If you decide to try this, please use the driver's Issues forum to ask questions about installation.

https://github.com/IntelRealSense/realsense_mipi_platform_driver/issues

---

An alternative backup plan you might be able to consider if the ROS2 approach does not work out is to switch to ROS1 and the RealSense ROS1 wrapper, as SLAM tends to be easier to implement in ROS1 than ROS2. Intel have a ROS1 / RTABMAP SLAM guide at the link below.

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i

[denysderihlazov]

I am not able to engage in communication activities such as screen-sharing, remote control or live-chat and can only provide advice through comments on this support forum, unfortunately.

JetPack 6 has problems with IMU-equipped RealSense cameras because JP6 removed a function called hiddraw that RealSense IMU cameras (which are known as HID devices) make use of. These problems can manifest as no IMU data available or the camera not being detected at all.

Would it be possible for you to change Ubuntu 22.04 to 20.04 and use JetPack 5.1.3 instead of JP6? RealSense IMU cameras should be able to work with JP 5.1.3.

If you need to use 22.04 and JP6 then IMU support for JP6 on a USB cable connection can be added by installing the RealSense MIPI driver at the link below, though it is a complex installation process.

https://github.com/IntelRealSense/realsense_mipi_platform_driver

If you decide to try this, please use the driver's Issues forum to ask questions about installation.

https://github.com/IntelRealSense/realsense_mipi_platform_driver/issues

An alternative backup plan you might be able to consider if the ROS2 approach does not work out is to switch to ROS1 and the RealSense ROS1 wrapper, as SLAM tends to be easier to implement in ROS1 than ROS2. Intel have a ROS1 / RTABMAP SLAM guide at the link below.

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i

Thank you so much for your advice and insights. I have already tried MIPI and I'm not sure will it work or not. Cause there are no step by step tutorials with how to make Intel Realsense D455 working with Jetpack 6+. My other scripts are based on Jetpack 6. So downgrade to JP 5+ could be a huge problem in the future.

Since I have two NVMe SSDs on my Jetson Orin NX, I’ll try installing JetPack 5.1.3 on one of them. This setup will limit me to using ROS 2 Galactic or ROS 2 Foxy.

I’m leaning towards ROS 2 Foxy, as its support ended in 2023, which might make it more stable than Galactic for my use case. However, I’m unsure which ROS 2 version would be the better choice for achieving SLAM/RTAB with the RealSense D455.

Would you happen to have a 100% reliable recommendation on which combination of JetPack and ROS to use to ensure the best stability and functionality for SLAM or RTAB? Ideally, I’d like to stay on ROS 2, as I’ve previously encountered issues trying to use ROS 1 with this camera.

This project is quite critical for me, as I’ve been stuck for over six months trying to solve this problem, and a working solution would mean a lot to me. Thank you in advance for any further guidance you can provide.

[MartyG-RealSense]

I do not have a recommendation for a specific JP version other than 5.1.3. However, RealSense ROS2 users on this support forum who have attempted to implement SLAM have often used a combination of slam_toolbox and depthimage_to_laserscan instead of RTABMAP.

https://github.com/IntelRealSense/realsense-ros/issues/3235#issuecomment-2447530896 is a currently open case that is using these two tools with 22.04 and Humble and an IMU-equipped D435i camera.

Below is a link to the ROS2 Humble version of slam_toolbox.

https://github.com/SteveMacenski/slam_toolbox/tree/humble

[denysderihlazov]

Hi Marty,

Thank you again for your previous guidance. Following your recommendation, I transitioned my setup to JetPack 5.1.3 with Ubuntu 20.04 and installed ROS2 Foxy. Here's a summary of the steps I have taken so far:

RealSense SDK: I installed the RealSense SDK version 2.55.1 and then downgraded to 2.51.1 to ensure compatibility with my Intel RealSense D455 camera. The ROS2 realsense2_camera package was also installed and configured.

SLAM Setup: I attempted to set up RTAB-Map using the rtabmap_ros package but encountered issues with the absence of certain files in the package (e.g., rtabmap.launch.py). To address this, I tried:
    Cloning the rtabmap_ros repository from multiple branches (foxy-devel, master, and noetic-devel).
    Manually creating and placing the rtabmap.launch.py file in the appropriate directory to initialize RTAB-Map.

Dependencies: While building rtabmap_ros, I encountered errors pointing to missing dependencies (librealsense2, rtabmap_costmap_plugins, and rtabmap_legacy). I cloned and attempted to build these packages individually.

Point Cloud and SLAM Tools: The camera is successfully publishing topics like /camera/camera/depth/image_rect_raw, but there were challenges in configuring slam_toolbox and depthimage_to_laserscan for SLAM processing.

Remaining Challenges:
    Despite these efforts, RTAB-Map setup is still incomplete. The rtabmap_ros package does not seem to include the required launch files or executables even after building.
    PointCloud2 data visualization in RViz2 is still problematic.

Could you please provide additional advice or resources to address these challenges? For example:

Are there known stable configurations for rtabmap_ros on ROS2 Foxy?
Should I prioritize a different approach, such as using slam_toolbox instead of RTAB-Map, as you previously mentioned?

Additionally, what tools or programs would you recommend for a more stable workflow? It seems many users face ongoing issues with RViz2 and PointCloud2. (Btw realsense-viewer works well and has no conflicts with ROS2 now. Also via Rviz2 I can still see image_raw and depth_img_raw, so the same as before)

What I've mentioned is that when I start camera topics via ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true

There are no errors and less warning messages: denys@ubuntu:~/ros2_ws$ ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true [INFO] [launch]: All log files can be found below /home/denys/.ros/log/2024-11-18-23-28-41-109571-ubuntu-36099 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [realsense2_camera_node-1]: process started with pid [36102] [realsense2_camera_node-1] [INFO] [1731968921.465484930] [camera.camera]: RealSense ROS v4.55.1 [realsense2_camera_node-1] [INFO] [1731968921.465695242] [camera.camera]: Built with LibRealSense v2.51.1 [realsense2_camera_node-1] [INFO] [1731968921.465729515] [camera.camera]: Running with LibRealSense v2.55.1

[realsense2_camera_node-1] [WARN] [1731968921.465806670] [camera.camera]: running with a different librealsense version [realsense2_camera_node-1] [WARN] [1731968921.465823246] [camera.camera]: than the one the wrapper was compiled with!

[realsense2_camera_node-1] [INFO] [1731968921.516918950] [camera.camera]: Device with serial number 234322305334 was found. [realsense2_camera_node-1] [realsense2_camera_node-1] [INFO] [1731968921.517061451] [camera.camera]: Device with physical ID /sys/devices/platform/3610000.xhci/usb2/2-1/2-1.3/2-1.3:1.3/video4linux/video0 was found. [realsense2_camera_node-1] [INFO] [1731968921.517088716] [camera.camera]: Device with name Intel RealSense D455 was found. [realsense2_camera_node-1] [INFO] [1731968921.517544924] [camera.camera]: Device with port number 2-1.3 was found. [realsense2_camera_node-1] [INFO] [1731968921.517586590] [camera.camera]: Device USB type: 3.2

[realsense2_camera_node-1] [INFO] [1731968921.518320504] [camera.camera]: JSON file is not provided [realsense2_camera_node-1] [INFO] [1731968921.518369305] [camera.camera]: Device Name: Intel RealSense D455 [realsense2_camera_node-1] [INFO] [1731968921.518394650] [camera.camera]: Device Serial No: 234322305334 [realsense2_camera_node-1] [INFO] [1731968921.518413275] [camera.camera]: Device physical port: /sys/devices/platform/3610000.xhci/usb2/2-1/2-1.3/2-1.3:1.3/video4linux/video0 [realsense2_camera_node-1] [INFO] [1731968921.518431868] [camera.camera]: Device FW version: 5.16.0.1 [realsense2_camera_node-1] [INFO] [1731968921.518448476] [camera.camera]: Device Product ID: 0x0B5C [realsense2_camera_node-1] [INFO] [1731968921.518464253] [camera.camera]: Sync Mode: Off [realsense2_camera_node-1] [INFO] [1731968921.739142719] [camera.camera]: Set ROS param depth_module.depth_profile to default: 848x480x30 [realsense2_camera_node-1] [INFO] [1731968921.741355022] [camera.camera]: Set ROS param depth_module.infra_profile to default: 848x480x30 [realsense2_camera_node-1] [WARN] [1731968921.785174179] [camera.camera]: Could not set param: rgb_camera.power_line_frequency with 3 Range: [0, 2]: parameter 'rgb_camera.power_line_frequency' could not be set: Parameter {rgb_camera.power_line_frequency} doesn't comply with integer range. [realsense2_camera_node-1] [INFO] [1731968921.800889201] [camera.camera]: Set ROS param rgb_camera.color_profile to default: 1280x720x30 [realsense2_camera_node-1] [INFO] [1731968921.802066011] [camera.camera]: Set ROS param gyro_fps to default: 200 [realsense2_camera_node-1] [INFO] [1731968921.802161502] [camera.camera]: Set ROS param accel_fps to default: 100 [realsense2_camera_node-1] [INFO] [1731968921.809748492] [camera.camera]: Stopping Sensor: Depth Module [realsense2_camera_node-1] [INFO] [1731968921.809952851] [camera.camera]: Stopping Sensor: RGB Camera [realsense2_camera_node-1] [INFO] [1731968921.815795619] [camera.camera]: Starting Sensor: Depth Module [realsense2_camera_node-1] [INFO] [1731968921.827763468] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 848, Height: 480, FPS: 30 [realsense2_camera_node-1] [INFO] [1731968922.008749452] [camera.camera]: Starting Sensor: RGB Camera [realsense2_camera_node-1] [INFO] [1731968922.023097770] [camera.camera]: Open profile: stream_type: Color(0), Format: RGB8, Width: 1280, Height: 720, FPS: 30 [realsense2_camera_node-1] [INFO] [1731968922.025633732] [camera.camera]: RealSense Node Is Up! ^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)

[realsense2_camera_node-1] [INFO] [1731968967.143310639] [camera.camera]: Stop Sensor: Depth Module [realsense2_camera_node-1] [INFO] [1731968967.143536375] [camera.camera]: Close Sensor. [realsense2_camera_node-1] [INFO] [1731968967.885164041] [camera.camera]: Close Sensor - Done. [realsense2_camera_node-1] [INFO] [1731968967.885241676] [camera.camera]: Stop Sensor: RGB Camera [realsense2_camera_node-1] [INFO] [1731968967.885261037] [camera.camera]: Close Sensor. [realsense2_camera_node-1] [INFO] [1731968968.739218144] [camera.camera]: Close Sensor - Done.

Your insights have been immensely valuable, and I hope to finalize this project soon. Thank you for your continued support.

[MartyG-RealSense]

The creator of the RealSense ROS1 wrapper once recommended at https://github.com/IntelRealSense/realsense-ros/issues/1080#issuecomment-590033913 to use the /voxel_cloud topic for the pointcloud instead of /camera/camera/depth/color/points when using rtabmap and also set Fixed Frame to map.

The log message running with a different librealsense version than the one the wrapper was compiled withindicates that the RealSense ROS wrapper was installed with one particular version of librealsense and then the librealsense version was changed afterwards. Each time a different librealsense version is installed, the ROS wrapper has to be installed again afterwards.

As rtabmap is what you are most familiar with, it may be worth persisting with this SLAM tool at least until the end of this week or until you are certain that you are not going to make any further progress and then consider changing to slam_toolbox. Setting a deadline for your changeover decision will help you to plan for the remaining time before your project deadline.

[denysderihlazov]

Hi MartyG,

Thank you for your continued support and guidance. Yesterday, I had RTAB-Map running (though the camera topics might not have been visible to RTAB-Map). However, today, I'm encountering a different issue. I haven’t changed the firmware or SDK version since my last message, but after finishing my attempts to make RTAB-Map work correctly with the RealSense D455, I shut down my Jetson Orin NX 16GB for the night. This morning, when I restarted my scripts, I encountered the following error:

Invalid Value in rs2_get_option(options:0xffff98037820, option:Auto Gain Limit Toggle): hwmon command 0x80( 5 0 0 0 ) failed (response -7= HW not ready)

Interestingly, the camera works fine with RealSense Viewer, but it doesn’t function properly when using ros2 launch.

I consulted ChatGPT for potential causes, and it suggested that the firmware and SDK versions might be incompatible. My current setup includes:

Firmware Version: 05.16.00.01
SDK Version: 2.51.1

ChatGPT and forums suggest that SDK 2.51.1 is best paired with firmware 05.13.00.50. I noticed similar recommendations from a few users on the forums.

However, I am hesitant to downgrade to an older firmware version, as I want to avoid introducing other issues. I had a similar concern when downgrading from SDK 2.55.1 to 2.51.1.

I have even considered switching to ORB-SLAM, but I prefer the visual clarity and user-friendliness of RTAB-Map for creating maps, which is essential for my project. I’ve spent hundreds of hours on this project without success, and even switched from JetPack 6.0 to 5.1.3 with Ubuntu 20.04.6 LTS to ensure compatibility. Unfortunately, the problem persists.

Could you please recommend a stable and not-too-outdated firmware version that is fully compatible with an SDK version you’re confident in? My ultimate goal is to use RTAB-Map with Intel RealSense D455.

If you have any kind of documentation or step-by-step guide, including commands and configuration tips, I would greatly appreciate it. I am running out of time and have already spent over six months and hundreds of hours troubleshooting this camera.

Thank you in advance for your help.:)

[MartyG-RealSense]

The Auto Gain Limit Toggle error usually occurs when there is a mismatch between the librealsense version and the firmware driver version (using an old firmware with a recent librealsense version or a recent firmware with an old librealsense version).

You are right to be concerned and careful about downgrading firmware, as a D455 will become unusable if a firmware older than 5.13.0.50 (the absolute minimum version supported by D455) is installed in the camera. As long as you do install 5.13.0.50 then you should be fine. I can appreciate why you are wary of making the change so close to your project deadline, but it may be a necessary change in order to achieve success.

[denysderihlazov]

Yes, the error has indeed disappeared. It’s strange that it appeared this morning, as it wasn’t present last night. But sometimes these things happen. I hope that switching to the 2022 firmware version (5.13.0.50) won’t limit the project in the future. I will now attempt to connect the RealSense D455 topics with RTAB-Map, and as I understand, these will later need to be visualized in RViz2.

Yesterday, when I was still using:

Firmware Version: 05.16.00.01
SDK Version: 2.51.1

I launched RTAB-Map and RViz2, and I could see the PointCloud2 topic and connect to it. However, as always, I couldn’t see anything on the map. It’s really unfortunate that there isn’t a comprehensive step-by-step tutorial—from setting up Ubuntu 22.04.6 to achieving a fully functional RTAB-Map integration.

By the way, I hope these errors are nothing serious:

22/11 16:43:25,985 ERROR [281472720140544] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 130 22/11 16:43:26,215 WARNING [281473058658560] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 22/11 16:43:27,523 WARNING [281473058658560] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 22/11 16:43:55,515 ERROR [281472938252544] (ds5-thermal-monitor.cpp:94) Error during thermal compensation handling: query is available during streaming only

I've checked that WARNING MSGs and as I can see it should not be a problem.

I’ve seen similar errors before, but I’m trying to eliminate all potential issues now to maximize my chances of success.

Thank you again:)

[MartyG-RealSense]

In general, control_transfer returned warnings can be safely ignored if the topics are publishing correctly.

Thermal compensation is not a significant problem either.

uvc streamer watchdog triggered on endpoint can be a more a serious problem, especially as there is not a clear reason for why it occurs and so it is difficult to debug as it is equal to a vague "something went wrong" message. But if the topics are performing normally then that message too can likely be safely disregarded if it is not causing your launch to stop and exit.

[denysderihlazov]

Hi Marty,

Thank you for your response and insights regarding the firmware and SDK version compatibility.

I wanted to share an update and get your expert opinion on a few issues I'm facing. First, I've switched to firmware version 5.13.0.50 and SDK version 2.51.1 on Ubuntu 20.04.6 LTS with ROS2 Foxy running on a Jetson Orin NX (16GB). The camera launches fine both in realsense-viewer and in ROS2, and I can see the following topics:

/camera/aligned_depth_to_color/camera_info /camera/aligned_depth_to_color/image_raw /camera/color/camera_info /camera/color/image_raw /camera/color/metadata /camera/depth/camera_info /camera/depth/color/points /camera/depth/image_rect_raw /camera/depth/metadata /camera/extrinsics/depth_to_color /camera/imu /parameter_events /rosout /tf_static

I’m currently attempting to integrate the RealSense D455 with RTAB-Map in ROS2 and later visualize the map in Rviz2. I’ve seen examples like this repository for the D435, which uses additional parameters during launch. My launch configuration looks like this:

ros2 launch realsense2_camera rs_launch.py \ align_depth.enable:=true \ pointcloud.enable:=true \ rgb_camera.enable:=true \ depth_module.enable:=true

This setup appears sufficient for RTAB-Map integration, as the required topics (e.g., /camera/color/image_raw, /camera/depth/image_rect_raw, and /camera/color/camera_info) are available. However, I suspect I’ll need to remap these in the RTAB-Map configuration to ensure proper functionality. New Issue with Rviz2

Yesterday, I could see the RGB and depth topics from the camera in Rviz2. However, after switching firmware and rebuilding both the SDK (librealsense) and the ROS2 Foxy workspace, these topics are no longer visible in Rviz2. I’m unsure what could have caused this change, as the topics are still listed via ros2 topic list. Concerns about Errors

During the launch, I encounter the following error, which seems to relate to USB:

(messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 130 error: Cannot allocate memory, number: 12 (dispatcher.cpp:34) Dispatcher [0xffff50778ac0] exception caught: failed to submit UVC request while start streaming

My setup:

The RealSense D455 is connected to the Jetson Orin NX via a USB 3.2 port.
No other devices are connected to the USB ports.
I’m using the original USB cable provided with the camera.

Questions for You

Criticality of USB Errors:
    Are these errors a sign of insufficient power or bandwidth issues, even though the Jetson Orin NX should have ample USB 3.2 capability?
    Would reducing resolution or frame rate (e.g., 640x480x30) help mitigate these errors, or do you recommend a different solution?

RTAB-Map Integration:
    Based on your experience, are there specific configurations or launch parameters you’d suggest to streamline the integration with RTAB-Map?

Missing Rviz2 Topics:
    Given that the RGB and depth topics are listed in ros2 topic list but no longer appear in Rviz2, could this be related to the changes in firmware, SDK, or ROS2 workspace rebuild? Are there any steps you would suggest to debug this?

I appreciate your time and expertise, as always. Let me know if additional logs or tests could help narrow down these issues.

Best regards, Denys

[denysderihlazov]

Following up on my previous message, I've made some progress, but I'm encountering a few new issues. Current Setup and Observations:

Realsense D455: After reinstalling the Realsense drivers, SDK, and rebuilding RTAB-Map, the camera now works better. I can see the RGB and Depth image topics in RViz2. However: Depth images are displayed correctly and update in real-time. RGB images, however, appear frozen in RViz2, even though they are published.

Launch Commands:

    For the camera:

  ros2 launch realsense2_camera rs_launch.py \
      align_depth.enable:=true \
      pointcloud.enable:=true \
      rgb_camera.enable:=true \
      depth_module.enable:=true \
      enable_sync:=true

For RTAB-Map:

ros2 launch rtabmap_ros rtabmap.launch.py \
    args:="--delete_db_on_start" \
    depth_topic:=/camera/depth/image_rect_raw \
    rgb_topic:=/camera/color/image_raw \
    camera_info_topic:=/camera/color/camera_info \
    approx_sync:=true \
    frame_id:=camera_link

I modeled the topic arguments after D435 examples and adjusted them for the D455.

Console Output:

When launching the camera, I see the following warning:

[WARN] [camera.camera]: XXX Hardware Notification: Left MIPI error,1.73233e+12,Error,Hardware Error

RTAB-Map does not receive any data from the camera topics, as shown in its logs.

Also here are topics I can see when camera is working via ros2 launch cmd:

    denys@ubuntu:~/ros2_ws$ ros2 topic list
    /camera/aligned_depth_to_color/camera_info
    /camera/aligned_depth_to_color/image_raw
    /camera/color/camera_info
    /camera/color/image_raw
    /camera/color/metadata
    /camera/depth/camera_info
    /camera/depth/color/points
    /camera/depth/image_rect_raw
    /camera/depth/metadata
    /camera/extrinsics/depth_to_color
    /camera/imu
    /parameter_events
    /rosout
    /tf_static
    denys@ubuntu:~/ros2_ws$ 

[MartyG-RealSense]

Do you need to publish the aligned depth topics with align_depth.enable please? If the pointcloud filter is enabled and depth and color is published then depth and color will automatically map together into an aligned pointcloud image without having to use align_depth. Not setting align_depth to true may reduce the processing burden on the Jetson's CPU (though having DBUILD_WITH_CUDA set to On, as you mentioned earlier that you do, does significantly reduce the CPU burden when using alignment or pointcloud).

[denysderihlazov]

Hi Marty,

Yes, I built the SDK with the DBUILD_WITH_CUDA=ON parameter. However, I am still facing issues with the camera and RTAB-Map. I’ll try to explain everything I’ve tested and what I’m observing right now. What I’ve done:

Launching the camera:
Initially, I launched the camera with the following parameters:

ros2 launch realsense2_camera rs_launch.py \ pointcloud.enable:=true \ rgb_camera.enable:=true \ depth_module.enable:=true \ align_depth.enable:=false \ enable_sync:=false \ depth_module.profile:=640x480x30 \ rgb_camera.profile:=640x480x30

But this didn’t help — the RGB image was still "frozen."

Then I tried launching the camera without passing any parameters:

ros2 launch realsense2_camera rs_launch.py

And surprisingly, the RGB image started working! However, now RTAB-Map doesn’t see the streams.

RTAB-Map: I configured RTAB-Map to subscribe to the corresponding topics:

ros2 launch rtabmap_ros rtabmap.launch.py \ args:="--delete_db_on_start" \ depth_topic:=/camera/aligned_depth_to_color/image_raw \ rgb_topic:=/camera/color/image_raw \ camera_info_topic:=/camera/color/camera_info \ approx_sync:=true \ frame_id:=camera_link

But RTAB-Map still doesn’t see any data from the camera.

Verifying data publication:

I reduced the resolution and frame rate:

depth_module.profile:=640x480x15 \ rgb_camera.profile:=640x480x15

This improved the situation — data started appearing, but I noticed strange values in the publication (lots of "zeros").

Verifying RGB publication:

ros2 topic hz /camera/color/image_raw

Output:

average rate: 29.026

This looks normal.

Verifying PointCloud:

ros2 topic hz /camera/depth/color/points

After the first cycle of publishing, I see this warning:

    [WARN] [camera.camera]: No stream match for pointcloud chosen texture Process - Color

    After that, the publication stops.

Questions and issues:

How to correctly launch the camera?
What parameters would you recommend for the Intel RealSense D455 to make RGB and PointCloud work reliably in ROS2?

PointCloud issue:
Why does the PointCloud publish only once and then stop with warnings? Could parameters like pointcloud.texture_stream be causing this issue?

RTAB-Map doesn’t see the data:
What parameters should be used with RTAB-Map for the D455? I suspect it might be related to stream alignment (align_depth).

Hardware MIPI errors:
Occasionally, I see this warning:

[WARN] [camera.camera]: XXX Hardware Notification: Left MIPI error

Could this be related to my platform (NVIDIA Jetson Orin NX 16GB), or is it an issue with the camera itself?

Why I’m writing:

I’ve been working on this system for a while now, and I believe many developers face similar problems. Unfortunately, there’s very little documentation available for D455, ROS2, and RTAB-Map integration. I plan to write a detailed guide to help others, but I need to resolve these issues first. Request for step-by-step guidance:

If you were setting up a D455 with ROS2 and RTAB-Map on NVIDIA Jetson Orin NX, could you please outline the exact steps you would follow, including any specific parameters or configurations to ensure everything works smoothly?

This would really help me debug my setup and validate my current approach.

I’d greatly appreciate any guidance or recommendations you can provide!

[MartyG-RealSense]

I do not have any experience with rtabmap_ros, so cannot offer a guide to compare to your own procedure, unfortunately. However, by searching for past cases involving RealSense + Jetson + ROS2 +rtabmap_ros, I did find a discussion at https://github.com/introlab/rtabmap_ros/issues/968 that might be helpful.

https://github.com/jdgalviss/jetbot-ros2 would also be worth looking at, as that does provide a step by step setup guide that uses ROS2, Jetson Nano, RealSense and rtabmap_ros.

There is also this Jetson Nano guide, which makes use of RealSense, ROS2 and rtabmap_ros, and additionally Docker:

https://www.codetomotion.com/blog/realsense-cameras-ros-2-foxy-and-docker-on-nvidia-jetson-nano/

---

In regard to your list of Questions and Issues:

How to correctly launch the camera? - in theory you should just need to launch rs_launch.py first and then the rtabmap_ros launch file second once the RealSense launch has completed. I would recommend doing these two launches in separate terminal windows.

PointCloud issue - if your RealSense launch has a problem with RGB where it is not supplying color frames though then on ROS2 I do not know what else you could do except enable the infra1 stream and set pointcloud.texture_stream to '1' to use the infrared stream to texture the pointcloud instead of the color stream.

RTAB-Map doesn’t see the data - as mentioned earlier, my knowledge of rtabmap is very limited.

My research of your case reminded me that rtabmap_ros provides a ROS2 launch file called realsense_d400.launch.py, which is launched with the command ros2 launch rtabmap_ros realsense_d400.launch.py

https://github.com/introlab/rtabmap_ros/issues/527

My recollection is that using this launch file makes using RealSense with rtabmap_ros easier as you only then need to use that one launch file instead of launching a separate rtabmap_ros launch file after the RealSense launch has completed.

Hardware Notification: Left MIPI error - this is usually related to an issue with the camera hardware. Sometimes it can mean that there is a problem with a cable called the 'interposer' inside the camera that joins its two circuit boards together. Sometimes though the error just happens for no apparent reason and then the camera works again afterwards, which it would not do if there was a persistent hardware related fault.