IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.58k stars 4.82k forks source link

Jetson orin nano /camera/imu #13201

Closed CSARobotics closed 2 months ago

CSARobotics commented 2 months ago

Required Info
Camera Model D435i
Firmware Version 5.16.0.1
Operating System & Version Linux (Ubuntu 20.04)
Kernel Version (Linux Only) 5.10
Platform NVIDIA Jetson Orin Nano
SDK Version legacy / 2.55.1.0
Language python
Segment Autonomous Robot

Issue Description

When I tried to use /camera/imu topic, any data does not come and also i couldn't visualize in rviz. Realsense-viewer send some error for motion module.

WhatsApp Image 2024-07-28 at 18 04 50 WhatsApp Image 2024-07-28 at 18 04 53 WhatsApp Image 2024-07-28 at 18 04 54

I tried to solve this problem with this issue solution. https://github.com/IntelRealSense/realsense-ros/issues/2563 I got this

https://github.com/user-attachments/assets/b8256df1-6baf-4086-b3f1-9058068bb719

MartyG-RealSense commented 2 months ago

Hi @CSARobotics When the control_transfer returned message generates continuously, it can indicate that there is a communication problem between the camera and the computer / computing board, typically an issue with the US port or the USB cable.

In the RealSense ROS wrapper, the IMU topics are disabled by default. They can be enabled and published in the /imu topic by adding the commands below to your ROS launch instruction.

ROS1 enable_gyro:=true enable_accel:=true unite_imu_method:=linear_interpolation

ROS2 enable_gyro:=true enable_accel:=true unite_imu_method:=2


May I also confirm please which JetPack version your Jetson is using. The IMU can work on Jetson if JetPack 5.1 is used. However, if JetPack 6 is used then a MIPI driver at the link below needs to be installed to add support for 'HID' devices (RealSense cameras equipped with an IMU).

https://github.com/IntelRealSense/realsense_mipi_platform_driver

If you are using JetPack 6 and you are unable to downgrade, or it is not possible to install the MIPI driver, then you can enable IMU support on a Jetson with JetPack 6 without the MIPI driver by building the librealsense SDK from source code with the libuvc backend build script at the link below. The libuvc method of building librealsense enables it to bypass the kernel.

https://github.com/IntelRealSense/librealsense/blob/master/doc/libuvc_installation.md

If librealsense is re-installed with this method then the ROS wrapper will need to be installed again afterwards.

MartyG-RealSense commented 2 months ago

Hi @CSARobotics Do you have an update about this case that you can provide, please? Thanks!

CSARobotics commented 2 months ago

Thank you for your interest. When I try it on my laptop with the same cable, there is no problem. So I don't think it's the cable. There does not appear to be a problem with ROS Wrapper. I think the problem is with the SDK. I installed the SDK using this repo https://github.com/jetsonhacks/installRealSenseSDK. There is no problem with RGB and Stereo camera, but I cannot view IMU data even with Realsense Viewer.

MartyG-RealSense commented 2 months ago

When using the installRealSenseSDK repo, did you install from packages with the installLibrealsense.sh script or build from source code with the buildLibrealsense.sh script?

The buildLibrealsense script should have a similar effect to using the libuvc backend procedure mentioned earlier in this discussion, as both the libuvc script and the buildLibrealsense script include the CMake build flag -DFORCE_LIBUVC=ON.

However, if you used installLibrealsense.sh to build from packages then the IMU would likely not be supported on JetPack 6 unless the RealSense MIPI driver was also installed.

CSARobotics commented 2 months ago

I used the buildLibrealsense script. The flags were as follows: /usr/bin/cmake ../ -DBUILD_EXAMPLES=true -DFORCE_LIBUVC=ON -DBUILD_WITH_CUDA="$USE_CUDA" -DCMAKE_BUILD_TYPE=release -DBUILD_PYTHON_BINDINGS=bool:true -DPYTHON_EXECUTABLE=$(which python3)

I'm currently using JetPack 6, but it doesn't matter to me. I can also switch to Jetpack 5 if it will work there.

MartyG-RealSense commented 2 months ago

Yes, I would strongly recommend trying JetPack 5.1 if you do not mind a downgrade of your JetPack.

CSARobotics commented 2 months ago

So, JetPack 5.1.2 or 5.1.3? Also, when I tried them with the buildLibrealsense script before, I got the same result. Is there an SDK installation guide where I can run IMU for JetPack 5?

MartyG-RealSense commented 2 months ago

I don't think that the version number (5.1.2 or 5.1.3) makes a difference, but people who have tried it have usually used 5.1.3. For example at https://github.com/IntelRealSense/librealsense/issues/12566#issuecomment-2205372168 a user with a Jetson Orin and JetPack 6 who installed with libuvc_installation.sh found that the IMU worked after downgrading to 5.1.3.

Another user at https://github.com/IntelRealSense/librealsense/issues/12807#issuecomment-2032658916 also had success with 5.1.3. They used a Jetson Nano guide at https://github.com/IntelRealSense/librealsense/issues/6964#issuecomment-707501049 for their source code installation.

One of my Intel RealSense colleagues explains at https://github.com/IntelRealSense/librealsense/issues/12807#issuecomment-2028612194 that the JetPack 6 issues are due to JP6 having no hidraw support (a RealSense camera equipped with an IMU is a 'HID' device).

CSARobotics commented 2 months ago

I'm currently flashing JetPack 5.1.3. I will try it as soon as it is finished and write the results.

CSARobotics commented 2 months ago

Realsense d435i became operational with all its functions.

Installation Method:

1 - Install Jetpack 5.1.3.

2 - Clone this repo https://github.com/jetsonhacks/installRealSenseSDK.

3 - Run installLibrealsense.sh

4 - Congratulations, the installation is completed.

Thank you very much for your help @MartyG-RealSense

MartyG-RealSense commented 2 months ago

You are very welcome, @CSARobotics - it's great to hear that JetPack 5.1.3 was a successful solution for you. Thanks very much for the update and for sharing your method!