Open mxmmbd1 opened 2 weeks ago
little update, I was able to make it run after these changes, but very slowly and doesn't work every time, as well extremely slow when doing rviz2 on my computer and going on topic /camera/camera/color/image_raw
documented here the steps since my first message:
Hardware: Intel RealSense D455 Camera Raspberry Pi 4 High-quality, short USB 3.2 cable connected to a high-power port
Software: Ubuntu 22.04 ROS2 Humble RealSense SDK 2.55.1 RealSense ROS2 Wrapper 4.55.1
First, ensure your system is updated and install dependencies required for RealSense SDK:
sudo apt update && sudo apt upgrade
sudo apt install git cmake libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev
We are using RealSense SDK v2.55.1, which you can download and install manually. Ensure you are in the home directory for these steps.
# Clone the RealSense SDK repository
git clone https://github.com/IntelRealSense/librealsense.git
cd librealsense
git checkout v2.55.1
# Build and install the SDK
mkdir build && cd build
cmake ../ -DCMAKE_BUILD_TYPE=Release
make -j4
sudo make install
It's important to use firmware version 5.16.0.1 for compatibility. Use the RealSense Viewer to check and update firmware:
realsense-viewer
In the viewer, navigate to the settings and update the firmware if necessary.
The RealSense ROS2 wrapper version 4.55.1 is used here. This wrapper integrates the RealSense SDK with ROS2, allowing you to access camera data in ROS2.
4.1. Create a ROS2 Workspace:
mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src
4.2. Clone the Wrapper Repository:
git clone https://github.com/IntelRealSense/realsense-ros.git -b ros2-4.55.1
4.3. Install ROS Dependencies:
cd ~/ros2_ws
sudo apt install python3-rosdep
sudo rosdep init
rosdep update
rosdep install --from-paths src --ignore-src -r -y
4.4. Build the ROS2 Wrapper:
cd ~/ros2_ws
colcon build --merge-install
4.5. Source the ROS2 Environment:
source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/local_setup.bash
Now that everything is installed, run the RealSense ROS2 node to start capturing data from the D455 camera.
ros2 run realsense2_camera realsense2_camera_node
Thanks so much @mxmmbd1 for sharing in detail the procedure that enabled you to get it working with your Raspberry Pi.
I note that you are experiencing extreme slowness, however. This is characteristic of using RealSense cameras with Raspberry Pi boards. Typically, the best that can be achieved is basic depth and color streaming.
Please test whether performance improves if you set depth and color to 640x480 resolution at 15 FPS.
ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=640x480x15 rgb_camera.color_profile:=640x480x15
Hi @mxmmbd1 Do you require further assistance with this case, please? Thanks!
Yes sorry for the late reply. After messaging that if had worked I tried to adjust various things to make the video quicker of /camera/camera/color/image_raw however I got to the point where I made the topic stop working. So I flashed the SD card again and started from scratch however I was not able to get back to the point that I had originally, currently I am not able to see the topics anymore
ubuntu@ubuntu:~$ ros2 run realsense2_camera realsense2_camera_node [INFO] [1731940021.978663215] [camera.camera]: RealSense ROS v4.55.1 [INFO] [1731940021.978969696] [camera.camera]: Built with LibRealSense v2.55.1 [INFO] [1731940021.979037659] [camera.camera]: Running with LibRealSense v2.55.1 [INFO] [1731940022.822715280] [camera.camera]: Device with serial number 239222300719 was found.
[INFO] [1731940022.822931150] [camera.camera]: Device with physical ID /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-2/2-2:1.0/video4linux/video0 was found. [INFO] [1731940022.822984928] [camera.camera]: Device with name Intel RealSense D455 was found. [INFO] [1731940022.823673667] [camera.camera]: Device with port number 2-2 was found. [INFO] [1731940022.823738926] [camera.camera]: Device USB type: 3.2
[INFO] [1731940022.824717054] [camera.camera]: JSON file is not provided [INFO] [1731940022.824800369] [camera.camera]: Device Name: Intel RealSense D455 [INFO] [1731940022.824846998] [camera.camera]: Device Serial No: 239222300719 [INFO] [1731940022.824888961] [camera.camera]: Device physical port: /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-2/2-2:1.0/video4linux/video0 [INFO] [1731940022.824936554] [camera.camera]: Device FW version: 5.13.0.55 [INFO] [1731940022.824978628] [camera.camera]: Device Product ID: 0x0B5C [INFO] [1731940022.825021276] [camera.camera]: Sync Mode: Off [WARN] [1731940022.980690335] [camera.camera]: Could not set param: rgb_camera.power_line_frequency with 3 Range: [0, 2]: parameter 'rgb_camera.power_line_frequency' could not be set: Parameter {rgb_camera.power_line_frequency} doesn't comply with integer range. [INFO] [1731940023.007753747] [camera.camera]: Stopping Sensor: Depth Module [INFO] [1731940023.008093932] [camera.camera]: Stopping Sensor: RGB Camera [INFO] [1731940023.008194246] [camera.camera]: Stopping Sensor: Motion Module [INFO] [1731940023.058745593] [camera.camera]: Starting Sensor: Depth Module [INFO] [1731940023.077059651] [camera.camera]: Open profile: stream_type: Infra(1), Format: Y8, Width: 848, Height: 480, FPS: 30 [INFO] [1731940023.077271687] [camera.camera]: Open profile: stream_type: Infra(2), Format: Y8, Width: 848, Height: 480, FPS: 30 [INFO] [1731940023.077381631] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 848, Height: 480, FPS: 30 [WARN] [1731940023.290064258] [camera.camera]: frame's time domain is HARDWARE_CLOCK. Timestamps may reset periodically. 18/11 14:27:08,148 ERROR [281472972286176] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:08,149 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:13,278 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:13,278 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:18,399 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:18,399 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:23,519 ERROR [281472980740320] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:23,520 ERROR [281472980740320] (rs.cpp:237) [rs2_get_video_stream_intrinsics( from:0xffff8006e5f0, intr:[ -1995988160x65535 p[-1.63268e-33 9.18341e-41] f[-1.6327e-33 9.18341e-41] UNKNOWN [9.18341e-41 -1.63294e-33 9.18341e-41 -1.63268e-33 9.18341e-41] ] ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out [WARN] [1731940043.520553839] [camera.camera]: No intrinsics available for this stream profile. Using zeroed intrinsics as default. 18/11 14:27:28,638 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:28,638 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:33,758 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:33,758 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:38,878 ERROR [281472972286176] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:38,878 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:43,998 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:43,998 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:49,117 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:49,118 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:54,238 ERROR [281472980740320] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:54,241 ERROR [281472980740320] (rs.cpp:237) [rs2_get_extrinsics( from:0xffff80041ef0, to:0xffff8006e5f0, extrin:[ r[1,0,0,0,1,0,0,0,1] t[0,0,0] ] ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out [ERROR] [1731940074.241663313] [camera.camera]: ./src/rs_node_setup.cpp:487:An exception has been thrown: std::exception [ERROR] [1731940074.241897164] [camera.camera]: ./src/rs_node_setup.cpp:369:An exception has been thrown: std::exception [ERROR] [1731940074.242089997] [camera.camera]: Error starting device: std::exception 18/11 14:27:59,357 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:59,358 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:04,478 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:04,478 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out ^C[INFO] [1731940086.528596824] [rclcpp]: signal_handler(signum=2) [INFO] [1731940086.530933517] [camera.camera]: Stop Sensor: Depth Module [INFO] [1731940086.531180553] [camera.camera]: Close Sensor. 18/11 14:28:09,599 ERROR [281472972286176] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:09,600 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:14,718 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:14,718 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:19,838 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:19,839 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:24,959 ERROR [281472972286176] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:24,959 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:30,079 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:30,079 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:35,199 ERROR [281472972286176] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:35,199 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out [INFO] [1731940115.300204153] [camera.camera]: Close Sensor - Done. 18/11 14:28:40,318 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:40,320 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
Ideally firmware driver version 5.16.0.1 should be used with librealsense 2.55.1 instead of firmware 5.13.0.55 (which was designed for librealsense 2.50.0 and 2.51.1).
Thank you for your last answer,
here's a sum up of what I did to get the stream back
We encountered challenges while updating the Intel RealSense D455 firmware on a Raspberry Pi running Ubuntu 22.04. Here are the steps we followed to resolve the issues and successfully verify the firmware:
Checked if the Intel RealSense D455 was recognized by the system:
lsusb
Output:
Bus 002 Device 005: ID 8086:0b5c Intel Corp. Intel(R) RealSense(TM) Depth Camera 455
Confirmed the connected device and its firmware version:
rs-fw-update -l
Output:
Connected devices:
1) [USB] Intel RealSense D455 s/n 239222300719, update serial number: 235523060312, firmware version: 5.16.0.1
Downloaded the firmware ZIP file directly:
wget https://downloadmirror.intel.com/821320/d400_series_fw_5_16_0_1.zip -O d400_fw.zip
Installed unzip
(if not already installed) and extracted the file:
sudo apt install unzip
unzip d400_fw.zip -d firmware
Verified the extracted .bin
file:
ls firmware/Signed_Image_UVC_5_16_0_1/
Output:
RealSense-D400-Series-Spec-Update.pdf
Signed_Image_UVC_5_16_0_1.bin
Ran the firmware update tool:
rs-fw-update -f firmware/Signed_Image_UVC_5_16_0_1/Signed_Image_UVC_5_16_0_1.bin
Encountered an error stating that the firmware version was already up to date.
Confirmed that the camera was functional using the RealSense Viewer:
realsense-viewer
Depth and RGB streams worked as expected.
The Intel RealSense D455 was already running the latest firmware version 5.16.0.1, so no update was required. The camera was successfully verified and is ready for use with librealsense and ROS2.
This document outlines the process followed after the firmware of the Intel RealSense D455 was verified as version 5.16.0.1. The goal was to set up the camera with ROS2 on a Raspberry Pi 4 running Ubuntu 22.04.
After confirming the firmware version:
rs-fw-update -l
Output:
Connected devices:
1) [USB] Intel RealSense D455 s/n 239222300719, update serial number: 235523060312, firmware version: 5.16.0.1
Ensured no conflicting versions of librealsense
existed:
sudo apt remove librealsense2* -y
sudo rm -rf /usr/local/lib/librealsense2.*
sudo rm -rf /usr/local/include/librealsense2
sudo rm -rf /usr/local/bin/rs-*
sudo rm /etc/udev/rules.d/99-realsense-libusb.rules
sudo rm /etc/apt/sources.list.d/librealsense.list
sudo apt update
Followed Intel's Linux Debian Installation Guide:
sudo apt-get install apt-transport-https
sudo mkdir -p /etc/apt/keyrings
curl -sSf https://librealsense.intel.com/Debian/librealsense.pgp | sudo tee /etc/apt/keyrings/librealsense.pgp > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/librealsense.pgp] https://librealsense.intel.com/Debian/apt-repo `lsb_release -cs` main" | \
sudo tee /etc/apt/sources.list.d/librealsense.list
sudo apt update
sudo apt install librealsense2-dkms librealsense2-utils librealsense2-dev librealsense2-dbg -y
Verified installation:
realsense-viewer
Ensured the ROS2 environment was correctly configured:
sudo apt install python3-rosdep -y
sudo rosdep init
rosdep update
mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src
Cloned the ROS2 wrapper for Intel RealSense:
git clone https://github.com/IntelRealSense/realsense-ros.git -b ros2-4.55.1
cd ~/ros2_ws
rosdep install --from-paths src --ignore-src -r -y
colcon build
Sourced the environment:
source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/local_setup.bash
Attempted to launch the RealSense node:
ros2 launch realsense2_camera rs_launch.py
Encountered errors related to high USB power demand and bandwidth issues.
Successfully streamed /camera/color/image_raw
by lowering resolution and frame rate:
ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=640x480x15 rgb_camera.color_profile:=640x480x15
Verified the topics:
ros2 topic list
Output:
/camera/color/image_raw
/camera/depth/image_rect_raw
...
Tested the camera functionality with adjusted parameters:
ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=640x480x15 rgb_camera.color_profile:=640x480x15
Verified camera streams in rviz2
:
rviz2
Added topics like /camera/color/image_raw
and /camera/depth/image_rect_raw
for visualization.
The Intel RealSense D455 camera was successfully set up and integrated with ROS2 on a Raspberry Pi running Ubuntu 22.04. Key steps included adjusting camera parameters to mitigate bandwidth issues and organizing the workspace for future use.
Thanks so much @mxmmbd1 for providing such a detailed guide for the benefit of members of the RealSense community!
Hi @mxmmbd1 After posting your detailed guide earlier, do you require further assistance with this case please? Thanks!
Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):
All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)
Issue Description
similar to issue #3158 @MartyG-RealSense @haidaumitom I am currently trying to connect the Realsense D455 with the Raspberry Pi 4 B over ROS 2. I used humble on the Ubuntu 22.04. I followed this guide https://github.com/IntelRealSense/realsense-ros?tab=readme-ov-file#installation-on-ubuntu. I followed the steps for the installation on Ubuntu and did Step 1. the guide of Humble. Step 2 I did option 2. Step 3 I did option 2. The camera is plugged into the USB 3.0 When trying to do ros2 launch realsense2_camera rs_launch.py ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=1280x720x30 pointcloud.enable:=true or ros2 run realsense2_camera realsense2_camera_node ros2 run realsense2_camera realsense2_camera_node --ros-args -p enable_color:=false -p spatial_filter.enable:=true -p temporal_filter.enable:=true
I get this response usually