ROS wrapper disabling camera detection

[PGannonGoddard]

Required Info
Camera Model	D435i
Firmware Version	5.13.05.55
Operating System & Version	Linux (Ubuntu 20.04)
Kernel Version (Linux Only)	5.15.136-tegra
Platform	Jetson Orin Nano Devkit
Librealsense SDK Version	2
Language	python
Segment	Robot
ROS Distro	Humble
RealSense ROS Wrapper Version	4.55.1

Issue Description

I've installed the librealsense library using this install script as mentioned in several of the issue threads. Once installed, I am able to successfully run realsense-viewer to view live feed from the camera.

However, after installing the ROS wrapper using sudo apt install ros-humble-realsense2-*, neither the ROS wrapper test node nor the original realsense-viewer are able to detect the attached camera. sudo realsense-viewer is still able to detect the camera at this stage.

Thanks for any assistance and I'm happy to provide any more information to help resolve this!

[MartyG-RealSense]

Hi @PGannonGoddard I note that you are using camera firmware driver version 5.13.05.55. If the libuvc backend script has installed SDK 2.55.1 then the firmware should be updated to 5.16.0.1.

[PGannonGoddard]

Thank you for your response @MartyG-RealSense, I've since updated the firmware to 5.16.0.1 using the realsense-viewer. I have not attempted to launch the ROS wrapper as of yet, as this update is now causing the motion module to fail while the RGB feed is still working prior to attempting to use the IMU. Upon loading realsense-viewer, I receive the following error message: notifications.cpp:511 - Motion Module force pause

Attempting to enable motion causes a stream of warnings to occur, as well as the following error messages which repeat continuously: global_timestamp_reader.cpp:239 - Error during time_diff_keeper polling: get_xu(ctrl=1) failed! Last Error: Success error-handling.cpp:99 - Error during polling error handler: get_xu(id=7) failed! Last Error: Success

After attempting to use the motion module, the camera needs to be power cycled in order to access the RGB feed once more.

[MartyG-RealSense]

Which JetPack version are you using, please?

[PGannonGoddard]

No problem: 6.0.

[MartyG-RealSense]

JetPack 6.0 requires that a RealSense MIPI driver at the link below be installed in order for RealSense cameras with an IMU to work with it.

https://github.com/IntelRealSense/realsense_mipi_platform_driver

The driver is not needed if you downgrade to JetPack 5.1.

If this is not possible for you then another fix that allows the IMU-equipped camera to work when JetPack 6 is installed without having to install the MIPI driver is to build librealsense from source code with the libuvc backend build script here:

https://github.com/IntelRealSense/librealsense/blob/master/doc/libuvc_installation.md

[PGannonGoddard]

The libuvc backend build script is what I had used to install the RealSense MIPI drivers. Prior to updating the firmware on the camera, I had confirmed all sensors were working through realsense_viewer. Would you recommend attempting to downgrade to JetPack 5.1 and trying again?

[MartyG-RealSense]

If you are able to downgrade to 5.1 then I would certainly recommend it as a possible fix for the IMU.

[PGannonGoddard]

Reviewing the other dependencies in the project, it doesn't look like downgrading to Jetpack 5.1/Ubuntu 20.04 will be possible.

[MartyG-RealSense]

For Nano boards, Intel strongly recommends enabling the barrel jack power connector (the DC power jack) using the instructions at the link below. Have you tried enabling your Orin Nano's barrel jack, please?

https://jetsonhacks.com/2019/04/10/jetson-nano-use-more-power/

[PGannonGoddard]

I have. I'm currently using a 19V 40W barrel jack supply with the Orin configured to run at 15W.

[MartyG-RealSense]

Let's remove the current installation of the ROS wrapper for the moment by using the command below to uninstall all packages with RealSense in their name. This should not affect your librealsense installation as that is based on a source code installation rather than packages.

dpkg -l | grep "realsense" | cut -d " " -f 3 | xargs sudo dpkg --purge

Are you then able to run realsense-viewer again and detect the camera without using sudo?

[PGannonGoddard]

I've confirmed that the ROS wrapper is not currently on the system with that command, and am getting the same results. Video feed is working with ~35% frame drops with a warning message upon boot of: notifications.cpp:511 Motion Module force pause Once the motion module is enabled all streams are no longer able to be accessed.

[MartyG-RealSense]

Are you enabling depth, RGB and Motion Module, or just RGB and Motion Module?

[PGannonGoddard]

Currently just the RGB and motion module.

[MartyG-RealSense]

Performance of the RGB may be better if the libuvc installation script is edited to add the flag -DBUILD_WITH_CUDA to line 46 if you have not done so already.

https://github.com/IntelRealSense/librealsense/blob/master/scripts/libuvc_installation.sh#L46

When this flag is added, librealsense will be built with support for CUDA enabled. This enables Jetson boards to automatically offload processing from the CPU onto the Nvidia graphics GPU on the Jetson board. librealsense's CUDA support accelerates pointclouds, depth-color alignment and YUY to RGB color conversion. It also automatically applies this acceleration to pointclouds, alignment and RGB color conversion in the ROS wrapper.

Motion Module Force Pause errors are more of a problem in the ROS wrapper than realsense-viewer. If it occurs in realsense-viewer then you can simply try disabling and re-enabling the IMU, but it is not as simple as that with a ROS launch (though you can enable and disable sensors during runtime in the ROS2 wrapper).

Have you tried adding initial_reset:=true to the ROS launch instruction to reset the camera at launch?

[PGannonGoddard]

I have not added that flag to the build script, and will do so next time I'm performing a build.

Regarding the motion module failure in the realsense viewer, once the IMU has been enabled, that is when the errors begin. Cycling the IMU does not resolve the issue as well, as all feeds become disabled until the camera itself is power cycled.

Prior to performing the firmware update on the camera, I had attempted to use the initial_reset:=true flag with no luck, and have not attempted to use the ROS wrapper since the IMU issues began post-update.

[MartyG-RealSense]

It is usually possible to display depth, RGB and IMU simultaneously without problems if the rs-capture SDK example program is used.

If the SDK has been installed with the example programs included then a pre-built executable version of rs-capture can be found in the usr/local/bin folder of Linux. Examples are included in the source code build by using the flags -DBUILD_EXAMPLES=TRUE -DBUILD_GRAPHICAL_EXAMPLES=TRUE

JetPack 6 is the complicating factor here though. If a libuvc installation is not allowing the IMU to work on a JetPack 6 equipped board (it usually does work) then the RealSense MIPI driver is the only other way of enabling IMU support on a JP6 machine.

Does your project require the IMU or are you attempting to test it simply because the camera has an IMU, please?

[MartyG-RealSense]

Hi @PGannonGoddard Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.