IntelRealSense / realsense-ros

ROS Wrapper for Intel(R) RealSense(TM) Cameras
http://wiki.ros.org/RealSense
Apache License 2.0
2.6k stars 1.76k forks source link

Not able to run Realsense D455 with ROS2 connected with Raspberry Pi 4B #3246

Open mxmmbd1 opened 2 weeks ago

mxmmbd1 commented 2 weeks ago

Required Info
Camera Model { D455 }
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version { Linux (Ubuntu 22) }
Kernel Version (Linux Only) 5.15.0-1064-raspi
Platform Raspberry Pi
Librealsense SDK Version { 2.55.1 }
Language {C/C#/labview/opencv/pcl/python/unity }
Segment {Robot}
ROS Distro {Humble }
RealSense ROS Wrapper Version {4.56.1}

Issue Description

similar to issue #3158 @MartyG-RealSense @haidaumitom I am currently trying to connect the Realsense D455 with the Raspberry Pi 4 B over ROS 2. I used humble on the Ubuntu 22.04. I followed this guide https://github.com/IntelRealSense/realsense-ros?tab=readme-ov-file#installation-on-ubuntu. I followed the steps for the installation on Ubuntu and did Step 1. the guide of Humble. Step 2 I did option 2. Step 3 I did option 2. The camera is plugged into the USB 3.0 When trying to do ros2 launch realsense2_camera rs_launch.py ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=1280x720x30 pointcloud.enable:=true or ros2 run realsense2_camera realsense2_camera_node ros2 run realsense2_camera realsense2_camera_node --ros-args -p enable_color:=false -p spatial_filter.enable:=true -p temporal_filter.enable:=true

I get this response usually

ubuntu@ubuntu:~$ ros2 run realsense2_camera realsense2_camera_node
[INFO] [1730965261.762033036] [camera.camera]: RealSense ROS v4.55.1
[INFO] [1730965261.762469221] [camera.camera]: Built with LibRealSense v2.55.1
[INFO] [1730965261.762566832] [camera.camera]: Running with LibRealSense v2.55.1
[INFO] [1730965262.656205988] [camera.camera]: Device with serial number 239222300719 was found.

[INFO] [1730965262.656402118] [camera.camera]: Device with physical ID /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-2/2-2:1.0/video4linux/video0 was found.
[INFO] [1730965262.656456747] [camera.camera]: Device with name Intel RealSense D455 was found.
[INFO] [1730965262.657174043] [camera.camera]: Device with port number 2-2 was found.
[INFO] [1730965262.657278394] [camera.camera]: Device USB type: 3.2
[INFO] [1730965262.660164317] [camera.camera]: getParameters...
[INFO] [1730965262.664702702] [camera.camera]: JSON file is not provided
[INFO] [1730965262.664834887] [camera.camera]: Device Name: Intel RealSense D455
[INFO] [1730965262.664899831] [camera.camera]: Device Serial No: 239222300719
[INFO] [1730965262.664961350] [camera.camera]: Device physical port: /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-2/2-2:1.0/video4linux/video0
[INFO] [1730965262.665021146] [camera.camera]: Device FW version: 5.13.0.55
[INFO] [1730965262.665077849] [camera.camera]: Device Product ID: 0x0B5C
[INFO] [1730965262.665132146] [camera.camera]: Sync Mode: Off
[WARN] [1730965262.820846782] [camera.camera]: Could not set param: rgb_camera.power_line_frequency with 3 Range: [0, 2]: parameter 'rgb_camera.power_line_frequency' could not be set: Parameter {rgb_camera.power_line_frequency} doesn't comply with integer range.
[INFO] [1730965262.850490900] [camera.camera]: Stopping Sensor: Depth Module
[INFO] [1730965262.850802678] [camera.camera]: Stopping Sensor: RGB Camera
[INFO] [1730965262.850898844] [camera.camera]: Stopping Sensor: Motion Module
[INFO] [1730965262.910490802] [camera.camera]: Starting Sensor: Depth Module
[INFO] [1730965262.928259561] [camera.camera]: Open profile: stream_type: Infra(1), Format: Y8, Width: 848, Height: 480, FPS: 30
[INFO] [1730965262.928471931] [camera.camera]: Open profile: stream_type: Infra(2), Format: Y8, Width: 848, Height: 480, FPS: 30
[INFO] [1730965262.928582987] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 848, Height: 480, FPS: 30
[WARN] [1730965263.142029565] [camera.camera]: frame's time domain is HARDWARE_CLOCK. Timestamps may reset periodically.
 07/11 07:41:07,957 ERROR [281472992733408] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:07,982 ERROR [281472992733408] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:13,085 ERROR [281472762112224] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
 07/11 07:41:13,086 ERROR [281472762112224] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
 07/11 07:41:18,206 ERROR [281472753658080] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:18,207 ERROR [281472753658080] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:23,326 ERROR [281473001187552] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:23,326 ERROR [281473001187552] (rs.cpp:237) [rs2_get_video_stream_intrinsics( from:0xffff8006daa0, intr:[ -1975540928x65535  p[-9.22703e-33 9.18341e-41]  f[-9.22711e-33 9.18341e-41]  UNKNOWN [9.18341e-41 -9.22806e-33 9.18341e-41 -9.22703e-33 9.18341e-41] ] ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
[WARN] [1730965283.327016359] [camera.camera]: No intrinsics available for this stream profile. Using zeroed intrinsics as default.
 07/11 07:41:28,445 ERROR [281472762112224] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
 07/11 07:41:28,445 ERROR [281472762112224] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
 07/11 07:41:33,565 ERROR [281472753658080] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:33,566 ERROR [281472753658080] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:38,686 ERROR [281472992733408] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:38,686 ERROR [281472992733408] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:43,805 ERROR [281472762112224] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
 07/11 07:41:43,806 ERROR [281472762112224] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
 07/11 07:41:48,925 ERROR [281472753658080] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:48,925 ERROR [281472753658080] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:54,046 ERROR [281473001187552] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
 07/11 07:41:54,047 ERROR [281473001187552] (rs.cpp:237) [rs2_get_extrinsics( from:0xffff800412b0, to:0xffff8006daa0, extrin:[ r[1,0,0,0,1,0,0,0,1]  t[0,0,0] ] ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out
[ERROR] [1730965314.047458440] [camera.camera]: ./src/rs_node_setup.cpp:487:An exception has been thrown: std::exception
[ERROR] [1730965314.047864754] [camera.camera]: ./src/rs_node_setup.cpp:369:An exception has been thrown: std::exception
[ERROR] [1730965314.048224493] [camera.camera]: Error starting device: std::exception
 07/11 07:41:59,165 ERROR [281472762112224] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
 07/11 07:41:59,165 ERROR [281472762112224] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory
mxmmbd1 commented 2 weeks ago

little update, I was able to make it run after these changes, but very slowly and doesn't work every time, as well extremely slow when doing rviz2 on my computer and going on topic /camera/camera/color/image_raw

documented here the steps since my first message:

Hardware: Intel RealSense D455 Camera Raspberry Pi 4 High-quality, short USB 3.2 cable connected to a high-power port

Software: Ubuntu 22.04 ROS2 Humble RealSense SDK 2.55.1 RealSense ROS2 Wrapper 4.55.1

  1. Install Dependencies

First, ensure your system is updated and install dependencies required for RealSense SDK:

sudo apt update && sudo apt upgrade
sudo apt install git cmake libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev
  1. Install the RealSense SDK (librealsense)

We are using RealSense SDK v2.55.1, which you can download and install manually. Ensure you are in the home directory for these steps.

# Clone the RealSense SDK repository
git clone https://github.com/IntelRealSense/librealsense.git
cd librealsense
git checkout v2.55.1

# Build and install the SDK
mkdir build && cd build
cmake ../ -DCMAKE_BUILD_TYPE=Release
make -j4
sudo make install
  1. Update Camera Firmware

It's important to use firmware version 5.16.0.1 for compatibility. Use the RealSense Viewer to check and update firmware:

realsense-viewer In the viewer, navigate to the settings and update the firmware if necessary.

  1. Install the RealSense ROS2 Wrapper

The RealSense ROS2 wrapper version 4.55.1 is used here. This wrapper integrates the RealSense SDK with ROS2, allowing you to access camera data in ROS2.

4.1. Create a ROS2 Workspace:

mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src

4.2. Clone the Wrapper Repository: git clone https://github.com/IntelRealSense/realsense-ros.git -b ros2-4.55.1 4.3. Install ROS Dependencies:

cd ~/ros2_ws
sudo apt install python3-rosdep
sudo rosdep init
rosdep update
rosdep install --from-paths src --ignore-src -r -y

4.4. Build the ROS2 Wrapper:

cd ~/ros2_ws
colcon build --merge-install

4.5. Source the ROS2 Environment:

source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/local_setup.bash
  1. Run the RealSense ROS2 Node

Now that everything is installed, run the RealSense ROS2 node to start capturing data from the D455 camera.

ros2 run realsense2_camera realsense2_camera_node

MartyG-RealSense commented 2 weeks ago

Thanks so much @mxmmbd1 for sharing in detail the procedure that enabled you to get it working with your Raspberry Pi.

I note that you are experiencing extreme slowness, however. This is characteristic of using RealSense cameras with Raspberry Pi boards. Typically, the best that can be achieved is basic depth and color streaming.

Please test whether performance improves if you set depth and color to 640x480 resolution at 15 FPS.

ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=640x480x15 rgb_camera.color_profile:=640x480x15

MartyG-RealSense commented 1 week ago

Hi @mxmmbd1 Do you require further assistance with this case, please? Thanks!

mxmmbd1 commented 1 week ago

Yes sorry for the late reply. After messaging that if had worked I tried to adjust various things to make the video quicker of /camera/camera/color/image_raw however I got to the point where I made the topic stop working. So I flashed the SD card again and started from scratch however I was not able to get back to the point that I had originally, currently I am not able to see the topics anymore

ubuntu@ubuntu:~$ ros2 run realsense2_camera realsense2_camera_node [INFO] [1731940021.978663215] [camera.camera]: RealSense ROS v4.55.1 [INFO] [1731940021.978969696] [camera.camera]: Built with LibRealSense v2.55.1 [INFO] [1731940021.979037659] [camera.camera]: Running with LibRealSense v2.55.1 [INFO] [1731940022.822715280] [camera.camera]: Device with serial number 239222300719 was found.

[INFO] [1731940022.822931150] [camera.camera]: Device with physical ID /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-2/2-2:1.0/video4linux/video0 was found. [INFO] [1731940022.822984928] [camera.camera]: Device with name Intel RealSense D455 was found. [INFO] [1731940022.823673667] [camera.camera]: Device with port number 2-2 was found. [INFO] [1731940022.823738926] [camera.camera]: Device USB type: 3.2

[INFO] [1731940022.824717054] [camera.camera]: JSON file is not provided [INFO] [1731940022.824800369] [camera.camera]: Device Name: Intel RealSense D455 [INFO] [1731940022.824846998] [camera.camera]: Device Serial No: 239222300719 [INFO] [1731940022.824888961] [camera.camera]: Device physical port: /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-2/2-2:1.0/video4linux/video0 [INFO] [1731940022.824936554] [camera.camera]: Device FW version: 5.13.0.55 [INFO] [1731940022.824978628] [camera.camera]: Device Product ID: 0x0B5C [INFO] [1731940022.825021276] [camera.camera]: Sync Mode: Off [WARN] [1731940022.980690335] [camera.camera]: Could not set param: rgb_camera.power_line_frequency with 3 Range: [0, 2]: parameter 'rgb_camera.power_line_frequency' could not be set: Parameter {rgb_camera.power_line_frequency} doesn't comply with integer range. [INFO] [1731940023.007753747] [camera.camera]: Stopping Sensor: Depth Module [INFO] [1731940023.008093932] [camera.camera]: Stopping Sensor: RGB Camera [INFO] [1731940023.008194246] [camera.camera]: Stopping Sensor: Motion Module [INFO] [1731940023.058745593] [camera.camera]: Starting Sensor: Depth Module [INFO] [1731940023.077059651] [camera.camera]: Open profile: stream_type: Infra(1), Format: Y8, Width: 848, Height: 480, FPS: 30 [INFO] [1731940023.077271687] [camera.camera]: Open profile: stream_type: Infra(2), Format: Y8, Width: 848, Height: 480, FPS: 30 [INFO] [1731940023.077381631] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 848, Height: 480, FPS: 30 [WARN] [1731940023.290064258] [camera.camera]: frame's time domain is HARDWARE_CLOCK. Timestamps may reset periodically. 18/11 14:27:08,148 ERROR [281472972286176] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:08,149 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:13,278 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:13,278 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:18,399 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:18,399 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:23,519 ERROR [281472980740320] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:23,520 ERROR [281472980740320] (rs.cpp:237) [rs2_get_video_stream_intrinsics( from:0xffff8006e5f0, intr:[ -1995988160x65535 p[-1.63268e-33 9.18341e-41] f[-1.6327e-33 9.18341e-41] UNKNOWN [9.18341e-41 -1.63294e-33 9.18341e-41 -1.63268e-33 9.18341e-41] ] ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out [WARN] [1731940043.520553839] [camera.camera]: No intrinsics available for this stream profile. Using zeroed intrinsics as default. 18/11 14:27:28,638 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:28,638 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:33,758 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:33,758 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:38,878 ERROR [281472972286176] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:38,878 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:43,998 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:43,998 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:49,117 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:49,118 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:54,238 ERROR [281472980740320] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:27:54,241 ERROR [281472980740320] (rs.cpp:237) [rs2_get_extrinsics( from:0xffff80041ef0, to:0xffff8006e5f0, extrin:[ r[1,0,0,0,1,0,0,0,1] t[0,0,0] ] ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out [ERROR] [1731940074.241663313] [camera.camera]: ./src/rs_node_setup.cpp:487:An exception has been thrown: std::exception [ERROR] [1731940074.241897164] [camera.camera]: ./src/rs_node_setup.cpp:369:An exception has been thrown: std::exception [ERROR] [1731940074.242089997] [camera.camera]: Error starting device: std::exception 18/11 14:27:59,357 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:27:59,358 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:04,478 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:04,478 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out ^C[INFO] [1731940086.528596824] [rclcpp]: signal_handler(signum=2) [INFO] [1731940086.530933517] [camera.camera]: Stop Sensor: Depth Module [INFO] [1731940086.531180553] [camera.camera]: Close Sensor. 18/11 14:28:09,599 ERROR [281472972286176] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:09,600 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:14,718 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:14,718 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:19,838 ERROR [281472736749792] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:19,839 ERROR [281472736749792] (d400-thermal-monitor.cpp:97) Error during thermal compensation handling: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:24,959 ERROR [281472972286176] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:24,959 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:30,079 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:30,079 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:35,199 ERROR [281472972286176] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out 18/11 14:28:35,199 ERROR [281472972286176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Connection timed out [INFO] [1731940115.300204153] [camera.camera]: Close Sensor - Done. 18/11 14:28:40,318 ERROR [281472745203936] (librealsense-exception.h:52) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory 18/11 14:28:40,320 ERROR [281472745203936] (error-handling.cpp:99) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such file or directory

MartyG-RealSense commented 1 week ago

Ideally firmware driver version 5.16.0.1 should be used with librealsense 2.55.1 instead of firmware 5.13.0.55 (which was designed for librealsense 2.50.0 and 2.51.1).

mxmmbd1 commented 6 days ago

Thank you for your last answer,

here's a sum up of what I did to get the stream back


Steps Taken to Update Intel RealSense D455 Firmware on Raspberry Pi 4

We encountered challenges while updating the Intel RealSense D455 firmware on a Raspberry Pi running Ubuntu 22.04. Here are the steps we followed to resolve the issues and successfully verify the firmware:


1. Verified Camera Connection

Checked if the Intel RealSense D455 was recognized by the system:

lsusb

Output:

Bus 002 Device 005: ID 8086:0b5c Intel Corp. Intel(R) RealSense(TM) Depth Camera 455

Confirmed the connected device and its firmware version:

rs-fw-update -l

Output:

Connected devices:
1) [USB] Intel RealSense D455 s/n 239222300719, update serial number: 235523060312, firmware version: 5.16.0.1

2. Downloaded and Extracted Firmware

  1. Downloaded the firmware ZIP file directly:

    wget https://downloadmirror.intel.com/821320/d400_series_fw_5_16_0_1.zip -O d400_fw.zip
  2. Installed unzip (if not already installed) and extracted the file:

    sudo apt install unzip
    unzip d400_fw.zip -d firmware
  3. Verified the extracted .bin file:

    ls firmware/Signed_Image_UVC_5_16_0_1/

    Output:

    RealSense-D400-Series-Spec-Update.pdf
    Signed_Image_UVC_5_16_0_1.bin

3. Attempted Firmware Update

Ran the firmware update tool:

rs-fw-update -f firmware/Signed_Image_UVC_5_16_0_1/Signed_Image_UVC_5_16_0_1.bin

Encountered an error stating that the firmware version was already up to date.


4. Verified Functionality

Confirmed that the camera was functional using the RealSense Viewer:

realsense-viewer

Depth and RGB streams worked as expected.


Conclusion

The Intel RealSense D455 was already running the latest firmware version 5.16.0.1, so no update was required. The camera was successfully verified and is ready for use with librealsense and ROS2.

mxmmbd1 commented 6 days ago

Documentation of Steps Post Firmware Update (Version 5.16.0.1)

This document outlines the process followed after the firmware of the Intel RealSense D455 was verified as version 5.16.0.1. The goal was to set up the camera with ROS2 on a Raspberry Pi 4 running Ubuntu 22.04.


1. Preparing the Environment

Verified Existing Setup

After confirming the firmware version:

rs-fw-update -l

Output:

Connected devices:
1) [USB] Intel RealSense D455 s/n 239222300719, update serial number: 235523060312, firmware version: 5.16.0.1

2. Installing librealsense SDK

Removed Previous Installations

Ensured no conflicting versions of librealsense existed:

sudo apt remove librealsense2* -y
sudo rm -rf /usr/local/lib/librealsense2.*
sudo rm -rf /usr/local/include/librealsense2
sudo rm -rf /usr/local/bin/rs-*
sudo rm /etc/udev/rules.d/99-realsense-libusb.rules
sudo rm /etc/apt/sources.list.d/librealsense.list
sudo apt update

Installed Latest librealsense SDK from Intel

Followed Intel's Linux Debian Installation Guide:

sudo apt-get install apt-transport-https
sudo mkdir -p /etc/apt/keyrings
curl -sSf https://librealsense.intel.com/Debian/librealsense.pgp | sudo tee /etc/apt/keyrings/librealsense.pgp > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/librealsense.pgp] https://librealsense.intel.com/Debian/apt-repo `lsb_release -cs` main" | \
sudo tee /etc/apt/sources.list.d/librealsense.list
sudo apt update
sudo apt install librealsense2-dkms librealsense2-utils librealsense2-dev librealsense2-dbg -y

Verified installation:

realsense-viewer

3. Setting Up ROS2 with RealSense

Installed ROS2 Dependencies

Ensured the ROS2 environment was correctly configured:

sudo apt install python3-rosdep -y
sudo rosdep init
rosdep update

Created ROS2 Workspace

mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src

Cloned RealSense ROS2 Wrapper

Cloned the ROS2 wrapper for Intel RealSense:

git clone https://github.com/IntelRealSense/realsense-ros.git -b ros2-4.55.1

Installed Dependencies for ROS2 Wrapper

cd ~/ros2_ws
rosdep install --from-paths src --ignore-src -r -y

Built the Workspace

colcon build

Sourced the environment:

source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/local_setup.bash

4. Testing the RealSense Node

Launched RealSense Node

Attempted to launch the RealSense node:

ros2 launch realsense2_camera rs_launch.py

Encountered errors related to high USB power demand and bandwidth issues.

Adjusted Camera Stream Parameters

Successfully streamed /camera/color/image_raw by lowering resolution and frame rate:

ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=640x480x15 rgb_camera.color_profile:=640x480x15

Verified the topics:

ros2 topic list

Output:

/camera/color/image_raw
/camera/depth/image_rect_raw
...

5. Final Test and Confirmation

Ran the RealSense Node Again

Tested the camera functionality with adjusted parameters:

ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=640x480x15 rgb_camera.color_profile:=640x480x15

Visualized Camera Streams

Verified camera streams in rviz2:

rviz2

Added topics like /camera/color/image_raw and /camera/depth/image_rect_raw for visualization.


Conclusion

The Intel RealSense D455 camera was successfully set up and integrated with ROS2 on a Raspberry Pi running Ubuntu 22.04. Key steps included adjusting camera parameters to mitigate bandwidth issues and organizing the workspace for future use.

MartyG-RealSense commented 6 days ago

Thanks so much @mxmmbd1 for providing such a detailed guide for the benefit of members of the RealSense community!

MartyG-RealSense commented 12 hours ago

Hi @mxmmbd1 After posting your detailed guide earlier, do you require further assistance with this case please? Thanks!