Closed arpaterson closed 1 year ago
I noticed I have /usr/local/lib/librealsense.so.2.51
.
It seems I used the -b
option git checkout -b v2.50.0
and created a local branch "v.2.50.0" pointing at the current HEAD (v2.51).
Cleaning up and using git checkout v2.50.0
...
... did not resolve this.
And neither did building with the RS_USB backend.
the rosdep/apt installed ros-noetic-realsense2-camera
and ros-noetic-librealsense2
work, but I need to build with CUDA, and preferably the native backend.
Hi @arpaterson If you install librealsense and the RealSense ROS wrapper together from packages with the wrapper's Method 1 instructions then the packages will be based on the RSUSB backend and not contain CUDA support. However, CUDA support accelerates just three types of function: color conversion, depth-color alignment and pointcloud. So unless you are setting align_depth to true or using the pointcloud filter, having CUDA support enabled in librealsense is unlikely to provide a performance benefit.
At https://github.com/IntelRealSense/realsense-ros/issues/2326 where the error undefined symbol: _ZN2cv3MatC1Ev occurred on a Jetson Xavier NX, a RealSense ROS user at https://github.com/IntelRealSense/realsense-ros/issues/2326#issuecomment-1107658481 suggested a solution that solved the error in that particular case.
I am currently setting align_depth true, although I am unsure if I will continue to do that. This ROS project uses an AGX orin now, but should also run on a nano.
Have you had the opportunity to try the solution at https://github.com/IntelRealSense/realsense-ros/issues/2326#issuecomment-1107658481 which states that it can work with RSUSB = false (native backend).
Whilst CUDA support is not included when installing from packages with the ROS wrapper's Method 1 instructions, it is included if you build librealsense and the ROS wrapper separately (librealsense from Jetson packages or source code, the ROS wrapper from source code). Instructions for installing librealsense from packages on Jetson are at the link below.
CUDA support is automatically included in the librealsense packages. If librealsense is built from source code with CMake then you can add CUDA support by including -DBUILD_WITH_CUDA=true in the CMake build instruction.
I will try that shortly, however from today until the 18th I will not have the equipment.
I will come back to this on my return
Thanks very much @arpaterson for the update. I look forward to your next report. Good luck!
To clarify:
In the jetson instructions, we are referred to the ubuntu instructions "Follow the Ubuntu installation guide to install the missing components and configuration items:"
for ubuntu20.04 the ubuntu guide suggests
sudo apt-get install libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev at
I think I missed this step before.
And then returning to the jetson instructions
sudo apt-get install git libssl-dev libusb-1.0-0-dev libudev-dev pkg-config libgtk-3-dev -y
Am I following that correctly?
is it necessary to install libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev at
?
My interpretation of the instructions would be as follows.
sudo apt-get install git libssl-dev libusb-1.0-0-dev libudev-dev pkg-config libgtk-3-dev
sudo apt-get install libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev at
./scripts/setup_udev_rules.sh
The installation instructions state that libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev are necessary if you want to build the SDK with support for OpenGL-enabled graphical examples, otherwise the core SDK will be configured for a headless environment without a graphics display.
OK, yes I picked that up after re-reading the instructions. I proceed with the packages installed for graphical examples.
I added:
find_package( OpenCV REQUIRED)
and modified:
include_directories(
include
${realsense2_INCLUDE_DIR}
${catkin_INCLUDE_DIRS}
${OpenCV_INCLUDE_DIRS}
)
target_link_libraries(${PROJECT_NAME}
${realsense2_LIBRARY}
${catkin_LIBRARIES}
${CMAKE_THREAD_LIBS_INIT}
#{OpenCV_LIBRARIES}
)
But I still have this error when launching rs_rgbd.
In the realsense2_camera build output everything looks normal except for:
-- Set runtime path of "/home/sacvp-user/catkin_ws/install/lib/librealsense2_camera.so" to ""
It is the right track though, OpenCV is not being linked.
My full realsense2_camera CMakeLists:
cmake_minimum_required(VERSION 2.8.3)
project(realsense2_camera)
add_compile_options(-std=c++11)
option(BUILD_WITH_OPENMP "Use OpenMP" OFF)
option(SET_USER_BREAK_AT_STARTUP "Set user wait point in startup (for debug)" OFF)
add_definitions(-D_CRT_SECURE_NO_WARNINGS)
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
find_package(catkin REQUIRED COMPONENTS
message_generation
nav_msgs
roscpp
sensor_msgs
std_msgs
std_srvs
nodelet
cv_bridge
image_transport
tf
ddynamic_reconfigure
diagnostic_updater
)
find_package(OpenCV REQUIRED)
if(BUILD_WITH_OPENMP)
find_package(OpenMP)
if(NOT OpenMP_FOUND)
message(FATAL_ERROR "\n\n OpenMP is missing!\n\n")
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS} -fopenmp")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_EXE_LINKER_FLAGS}")
endif()
endif()
if(SET_USER_BREAK_AT_STARTUP)
message("GOT FLAG IN CmakeLists.txt")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DBPDEBUG")
endif()
if (WIN32)
find_package(realsense2 CONFIG REQUIRED)
else()
find_package(realsense2 2.50.0)
endif()
if(NOT realsense2_FOUND)
message(FATAL_ERROR "\n\n Intel RealSense SDK 2.0 is missing, please install it from https://github.com/IntelRealSense/librealsense/releases\n\n")
endif()
if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release)
endif()
string(TOUPPER "${CMAKE_BUILD_TYPE}" uppercase_CMAKE_BUILD_TYPE)
if (${uppercase_CMAKE_BUILD_TYPE} STREQUAL "RELEASE")
message(STATUS "Create Release Build.")
set(CMAKE_CXX_FLAGS "-O2 ${CMAKE_CXX_FLAGS}")
else()
message(STATUS "Create Debug Build.")
endif()
if (WIN32)
else()
set(CMAKE_CXX_FLAGS "-fPIE -fPIC -std=c++11 -D_FORTIFY_SOURCE=2 -fstack-protector -Wformat -Wformat-security -Wall ${CMAKE_CXX_FLAGS}")
endif()
add_message_files(
FILES
IMUInfo.msg
Extrinsics.msg
Metadata.msg
)
add_service_files(
FILES
DeviceInfo.srv
)
generate_messages(
DEPENDENCIES
sensor_msgs
std_msgs
)
set(CMAKE_NO_SYSTEM_FROM_IMPORTED true)
include_directories(
include
${realsense2_INCLUDE_DIR}
${catkin_INCLUDE_DIRS}
${OpenCV_INCLUDE_DIRS}
)
# RealSense ROS Node
catkin_package(
LIBRARIES ${PROJECT_NAME}
CATKIN_DEPENDS message_runtime roscpp sensor_msgs std_msgs
nodelet
cv_bridge
image_transport
ddynamic_reconfigure
nav_msgs
)
add_library(${PROJECT_NAME}
include/constants.h
include/realsense_node_factory.h
include/base_realsense_node.h
include/t265_realsense_node.h
src/realsense_node_factory.cpp
src/base_realsense_node.cpp
src/t265_realsense_node.cpp
)
add_dependencies(${PROJECT_NAME} ${PROJECT_NAME}_generate_messages_cpp)
add_dependencies(${PROJECT_NAME} ${catkin_EXPORTED_TARGETS})
target_include_directories(${PROJECT_NAME}
PRIVATE ${realsense2_INCLUDE_DIR})
target_link_libraries(${PROJECT_NAME}
${realsense2_LIBRARY}
${catkin_LIBRARIES}
${CMAKE_THREAD_LIBS_INIT}
#{OpenCV_LIBRARIES}
)
if(WIN32)
set_target_properties(${realsense2_LIBRARY} PROPERTIES MAP_IMPORTED_CONFIG_RELWITHDEBINFO RELEASE)
target_link_libraries(${PROJECT_NAME}
realsense2::realsense2
realsense2::realsense-file
)
endif()
# Install nodelet library
install(TARGETS ${PROJECT_NAME}
ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
)
# Install header files
install(DIRECTORY include/
DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
)
# Install launch files
install(DIRECTORY launch/
DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}/launch
)
# Install rviz files
install(DIRECTORY rviz/
DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}/rviz
)
# Install xml files
install(FILES nodelet_plugins.xml
DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
)
librealsense is built from source (realsense-viewer works) apt ros-noetic-librealsense and ros-noetic-realsense2* are not installed. realsense2_camera is in my ros catkin workspace
opencv4.5.0 is built from source and the existing opencv from jetpack5 has not been replaced/removed.
Perhaps it is something to do with this?
The RealSense ROS wrapper does not support an RGBD launch by default and support for it has to be installed first before using the wrapper's rs_rgbd.launch launch file. Can you confirm if you have installed RGBD support, please? It can be done on Noetic using the instruction below.
sudo apt-get install ros-noetic-rgbd-launch
Yes, I have installed rgbd-launch
Anthony Paterson
On 6. Sep 2022, at 06:38, MartyG-RealSense @.***> wrote:
The RealSense ROS wrapper does not support an RGBD launch by default and support for it has to be installed first before using the wrapper's rs_rgbd.launch launch file. Can you confirm if you have installed RGBD support, please? It can be done on Noetic using the instruction below.
sudo apt-get install ros-noetic-rgbd-launch
It was established earlier in this discussion at https://github.com/IntelRealSense/realsense-ros/issues/2467#issuecomment-1237303296 that librealsense and the RealSense Viewer were working correctly on Jetson Orin. However, the ROS wrapper has the undefined symbol error when launching.
As you suggest in the above link, it may be related to installing OpenCV 4.5.0 from source, since OpenCV installation is not a required step in the ROS wrapper install instructions.
Hi @arpaterson Do you require further assistance with this case, please? Thanks!
Case closed due to no further comments received.
Is it possible to reopen this?
Yes, it is fine to re-open it if you wish to resume the case.
Are there any requirements on opencv?
I have built openCV with CUDA, and it appears correctly in python3, jtop shows
OpenCV: compiled CUDA: YES
.
The only thing I can see is the openGL support is not there "NO" in openCV build information.
General configuration for OpenCV 4.5.0 =====================================
Version control: 4.5.0
Extra modules:
Location (extra): /home/sacvp-user/git/opencv/opencv_contrib/modules
Version control (extra): 4.5.0
Platform:
Timestamp: 2022-09-05T09:30:27Z
Host: Linux 5.10.104-tegra aarch64
CMake: 3.16.3
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/make
Configuration: RELEASE
CPU/HW features:
Baseline: NEON FP16
required: NEON
disabled: VFPV3
C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: /usr/bin/c++ (ver 9.4.0)
C++ flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG
C++ flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG
C Compiler: /usr/bin/cc
C flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG
C flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG
Linker flags (Release): -Wl,--gc-sections -Wl,--as-needed
Linker flags (Debug): -Wl,--gc-sections -Wl,--as-needed
ccache: NO
Precompiled headers: NO
Extra dependencies: m pthread cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda-11.4/lib64 -L/usr/lib/aarch64-linux-gnu
3rdparty dependencies:
OpenCV modules:
To be built: alphamat aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
Disabled: python2 world
Disabled by dependency: -
Unavailable: cnn_3dobj cvv hdf java js julia matlab ovis sfm ts viz
Applications: apps
Documentation: NO
Non-free algorithms: YES
GUI:
GTK+: YES (ver 3.24.20)
GThread : YES (ver 2.64.6)
GtkGlExt: NO
OpenGL support: NO
VTK support: NO
Media I/O:
ZLib: /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
JPEG: /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
WEBP: build (ver encoder: 0x020f)
PNG: /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.37)
TIFF: build (ver 42 - 4.0.10)
JPEG 2000: build (ver 2.3.1)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES
Video I/O:
DC1394: YES (2.2.5)
FFMPEG: YES
avcodec: YES (58.54.100)
avformat: YES (58.29.100)
avutil: YES (56.31.100)
swscale: YES (5.5.100)
avresample: YES (4.0.0)
GStreamer: YES (1.16.3)
v4l/v4l2: YES (linux/videodev2.h)
Parallel framework: TBB (ver 2020.1 interface 11101)
Trace: YES (with Intel ITT)
Other third-party libraries:
Lapack: YES (/usr/lib/aarch64-linux-gnu/liblapack.so /usr/lib/aarch64-linux-gnu/libcblas.so /usr/lib/aarch64-linux-gnu/libatlas.so)
Eigen: YES (ver 3.3.7)
Custom HAL: YES (carotene (ver 0.0.1))
Protobuf: build (3.5.1)
NVIDIA CUDA: YES (ver 11.4, CUFFT CUBLAS FAST_MATH)
NVIDIA GPU arch: 87
NVIDIA PTX archs:
cuDNN: YES (ver 8.4.1)
Python 3:
Interpreter: /usr/bin/python3 (ver 3.8.10)
Libraries: /usr/lib/aarch64-linux-gnu/libpython3.8.so (ver 3.8.10)
numpy: /usr/lib/python3/dist-packages/numpy/core/include (ver 1.17.4)
install path: lib/python3.8/dist-packages/cv2/python-3.8
Python (for build): /usr/bin/python2.7
Java:
ant: NO
JNI: NO
Java wrappers: NO
Java tests: NO
Install to: /usr/local
-----------------------------------------------------------------
Support for OpenGL graphics in librealsense on Jetson would be enabled by either building librealsense from source code with CMake and including the build term -DBUILD_GRAPHICAL_EXAMPLES=true, or building from packages and installing the 'librealsense-utils' package with the instruction sudo apt-get install librealsense2-utils
A non-RealSense discussion at https://github.com/mdegans/nano_build_opencv/issues/51 provides advice about building OpenCV on Jetson Nano with OpenGL support.
I think I've solved this through a combination of
I don't have any clear learnings to share, but the problem is solved.
For the record heres what I did
undefined reference to glXGetProcAddressARB()
)Additionally, I
It is possible that between having apt installed versions, workspace sourcing, etc that I was not running the compiled version of librealsense.
Anyay, the changes to realsense-ros CMakeLists.txt works.
Just before closing this, @MartyG-RealSense Do you have any idea what performance change/how much GPU usage I should see? I'm not sure how to confirm that the Orin GPU is being utilized for the D455, and not just for desktop rendering.
I see almost none, and low framerates when connected over xrdp, but that is expected.
Connected to a monitor I see a solid 30FPS in rviz, up to 45 if uncapped.
That's excellent news that you achieved a solution. Thanks so much for sharing the details with the RealSense ROS community!
Below are some example figures for CPU vs GPU performance when using alignment with and without CUDA. The figures are a few years old but should provide a rough guide for expectations of reduction in CPU usage with CUDA.
At a recent case at https://github.com/IntelRealSense/librealsense/issues/10911 a RealSense ROS user who was checking whether librealsense's CUDA support was enabled reported that if the GPU was being utilized then it would be registered in nvtop.
I will have a chance to look at nvtop this weekend. If I do not see the expected resource use I will reopen, or open another issue.
Thank you again!
You are very welcome. I look forward to your next report. Good luck!
Hi @arpaterson Do you have an update about this case that you can provide, please? Thanks!
Case closed due to no further comments received.
nothing further at this point.
cmake_minimum_required(VERSION 2.8.3) project(realsense2_camera) add_compile_options(-std=c++11)
option(BUILD_WITH_OPENMP "Use OpenMP" OFF) option(SET_USER_BREAK_AT_STARTUP "Set user wait point in startup (for debug)" OFF)
find_package(OpenCV REQUIRED) find_package(catkin REQUIRED COMPONENTS message_generation nav_msgs roscpp sensor_msgs std_msgs nodelet cv_bridge image_transport tf ddynamic_reconfigure diagnostic_updater OpenCV REQUIRED )
if(BUILD_WITH_OPENMP) find_package(OpenMP) if(NOT OpenMP_FOUND) message(FATAL_ERROR "\n\n OpenMP is missing!\n\n") else() set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS} -fopenmp") set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_EXE_LINKER_FLAGS}") endif() endif()
if(SET_USER_BREAK_AT_STARTUP) message("GOT FLAG IN CmakeLists.txt") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DBPDEBUG") endif()
find_package(realsense2 2.29.0) if(NOT realsense2_FOUND) message(FATAL_ERROR "\n\n Intel RealSense SDK 2.0 is missing, please install it from https://github.com/IntelRealSense/librealsense/releases\n\n") endif()
if (CMAKE_BUILD_TYPE EQUAL "RELEASE") message(STATUS "Create Release Build.") set(CMAKE_CXX_FLAGS "-O2 ${CMAKE_CXX_FLAGS}") else() message(STATUS "Create Debug Build.") endif()
set(CMAKE_CXX_FLAGS "-fPIE -fPIC -std=c++11 -D_FORTIFY_SOURCE=2 -fstack-protector -Wformat -Wformat-security -Wall ${CMAKE_CXX_FLAGS}")
add_message_files( FILES IMUInfo.msg Extrinsics.msg )
generate_messages( DEPENDENCIES sensor_msgs std_msgs )
set(CMAKE_NO_SYSTEM_FROM_IMPORTED true) include_directories( include ${realsense2_INCLUDE_DIR} ${catkin_INCLUDE_DIRS} )
catkin_package( LIBRARIES ${PROJECT_NAME} CATKIN_DEPENDS message_runtime roscpp sensor_msgs std_msgs nodelet cv_bridge image_transport ddynamic_reconfigure nav_msgs )
add_library(${PROJECT_NAME} include/constants.h include/realsense_node_factory.h include/base_realsense_node.h include/t265_realsense_node.h src/realsense_node_factory.cpp src/base_realsense_node.cpp src/t265_realsense_node.cpp )
add_dependencies(${PROJECT_NAME} ${PROJECT_NAME}_generate_messages_cpp) add_dependencies(${PROJECT_NAME} ${catkin_EXPORTED_TARGETS})
target_include_directories(${PROJECT_NAME} PRIVATE ${realsense2_INCLUDE_DIR})
target_link_libraries(${PROJECT_NAME} ${realsense2_LIBRARY} ${catkin_LIBRARIES} ${CMAKE_THREAD_LIBS_INIT} )
install(TARGETS ${PROJECT_NAME} ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION} LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION} RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION} )
install(DIRECTORY include/ DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION} )
install(DIRECTORY launch/ DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}/launch )
install(DIRECTORY rviz/ DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}/rviz )
install(FILES nodelet_plugins.xml DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION} )
I have done a source build of
realsense-ros
andlibrealsense
(native backend + CUDA, but SKIPPING the kernel patches) on a nvidia AGX Orin. It was suggested here that this might work. I am gettingundefined symbol: _ZN2cv3MatC1Ev
when trying to launch.Output of
uname -a
Output with launching
rs_rgbd.launch
orrs_camera.launch
I think this might be related.