search

IntelRealSense / realsense-ros

ROS Wrapper for Intel(R) RealSense(TM) Cameras
http://wiki.ros.org/RealSense
Apache License 2.0
2.51k stars 1.74k forks source link

Installing Driver (and SDK) from ROS Dependency with CUDA Support for Jetsons #2303

Closed jyang-cpr closed 2 years ago

jyang-cpr commented 2 years ago

Clearpath Robotics' Dingo robot's ROS package(s) currently depend on realsense2_camera, as shown here: https://github.com/dingo-cpr/dingo_robot/blob/noetic-devel/dingo_bringup/package.xml#L15.

When the Dingo ROS package(s) are installed, the realsense2_camera driver will be installed automatically as a dependency. From my understanding of Method 1, this will also automatically install the librealsense2 SDK version without CUDA support enabled. This becomes a problem when the Dingo ROS package(s) are installed on a Jetson computer, since the automatically installed versions of the SDK and driver do not support CUDA.

I am currently experiencing this issue on a robot with a Jetson Nano, which has severely limited FPS/performance due to the GPU not being used/CUDA not supported. The workaround was to completely uninstall the versions of realsense2_camera and librealsense2 that were installed as ROS dependencies, and reinstall both using Method 2, finally leveraging the GPU and giving better FPS.

The workaround above works, but adds overhead to ROS development on Jetson robots with ROS packages depending on realsense2_camera. Aside from removing the dependency on realsense2_camera from package.xml and manually using Method 2 for all Jetson robots in the future, is there another (better) option?

MartyG-RealSense commented 2 years ago

Hi @jyang-cpr The development focus of the RealSense ROS wrapper is now on the 4.x versions for ROS2. So new feature additions to the ROS1 wrapper for Kinetic, Melodic and Noetic will not be implemented. As an example of this, support for the new RealSense D405 camera model that the 4.x wrapper already has is not planned to be added to the ROS1 wrapper.

Adding CUDA support for Jetson via Method 2 (building librealsense and the ROS1 wrapper separately instead of together from packages) is therefore likely to be the best option for you.

You could though investigate whether the RealSense ROS installation script 'installRealSenseROS.sh' provided by JetsonHacks still works (it is a few years old now) to see whether it could install modern ROS wrapper versions with Method 2 after some version number edits to the script whilst reducing overall installation time.

https://github.com/JetsonHacksNano/installRealSenseROS

https://github.com/JetsonHacksNano/installRealSenseROS/blob/master/installRealSenseROS.sh

You could potentially create a fork of the original JetsonHacks installRealSenseROS project GitHub above and then edit the script at your forked version and use that for Jetson installations.

jyang-cpr commented 2 years ago

Thanks for the response @MartyG-RealSense. I will look into possibly leveraging the JetsonHacks scripts, or some other way automating this process.

Can I confirm that Method 2 should be performed not only for Jetson computers, but for any computer (e.g. amd64) with an NVIDIA GPU (with CUDA)?

Also, I may be reaching but will there be plans in the ROS2 version to automate the installation of the SDK with/without CUDA support? For example, when installing the ROS driver and SDK using debian packages, to possibly detect the presence of an NVIDIA GPU and correspondingly installing the version of the SDK with CUDA support enabled?

MartyG-RealSense commented 2 years ago

The installation of the ROS wrapper in Method 2 is not involved in adding CUDA support. That is done when first building the librealsense SDK beforehand, as the CUDA support is part of librealsense. CUDA is exclusive to Nvidia graphics GPUs, so a non-Jetson computer such as a PC would need to have Nvidia graphics in some form, either built-in or as a video card.

Building librealsense with CUDA support should make it automatically applied to any of the three types of librealsense function that can take advantage of CUDA acceleration (color conversion, depth to color alignment and pointcloud generation). So it is not only the ROS wrapper that can benefit from CUDA support but also librealsense applications that use color conversion, depth-color alignment or the pointcloud processing block (align_to).

The librealsense SDK also has an alternative graphics acceleration system called GLSL processing blocks. It is 'vendor neutral', meaning that any GPU brand can use it, not just Nvidia, though there may not be a noticable improvement on computing devices with a low-end configuration. Support for GLSL in librealsense scripts can be added by putting gl:: in front of rs2:: instructions, so that for example, rs2::align and rs2::pointcloud becomes rs2::gl::align and rs2::gl::pointcloud

https://github.com/IntelRealSense/librealsense/pull/3654 has a very good pros and cons analysis of GLSL processing blocks and when it can be used.

Regarding your ROS2 wrapper suggestion, I am not involved in wrapper development as I am a RealSense Support Engineer, so I do not have information about future plans for CUDA in the ROS2 wrapper.

jyang-cpr commented 2 years ago

Thanks for the detailed answers. I will close this ticket now.