Adding OpenCV's GPU GFTT/ OpticalFlow and CudaSift support

matlabbe commented 3 months ago

These new features are available if built with OpenCV CUDA.

New features:

Faster GFTT: with GFTT/Gpu=true
Faster optical flow based F2F Odometry: with Vis/CorFlowGpu=true
Faster stereo correspondences: with Stereo/Gpu=true
Faster SIFT features extraction (with new CudaSift dependency): with SIFT/Gpu=true
Faster F2F Odometry approach using parameters above like this: --Odom/Strategy 1 --Vis/CorFlowGpu true --Vis/CoreType 1 --Stereo/Gpu true --GFTT/Gpu true --Vis/BundleAdjustment 0

Use this fork to build CudaSift: https://github.com/matlabbe/CudaSift

for issue https://github.com/Celebrandil/CudaSift/issues/90, a workaround has been implemented in this commit https://github.com/introlab/rtabmap/pull/1330/commits/74305397cfff0b5c419e0884abc529fb3bfdee97

matlabbe commented 2 months ago

When building rtabmap_ros against rtabmap with this PR (on Ubuntu 20.04 ROS Noetic with local version of OpenCV 4.2+Cuda shadowing OpenCV system binaries), we can get this error:

/usr/bin/ld: /usr/local/lib/libopencv_cudafeatures2d.so.4.2.0: undefined reference to `cv::cuda::device::ThrustAllocator::getAllocator()'
/usr/bin/ld: /usr/local/lib/libopencv_cudafeatures2d.so.4.2.0: undefined reference to `cv::cuda::StreamAccessor::getStream(cv::cuda::Stream const&)'

It seems to be because during linking (see with make VERBOSE=1), the binary /usr/lib/x86_64-linux-gnu/libopencv_core.so.4.2.0 is in the list (because of ros dependency cv_bridge) before /usr/local/lib/libopencv_core.so.4.2.0, so cuda core functions are not found. Well, a workaround if we don't want to rebuild cv_bridge is to make a symlink from /usr/lib/x86_64-linux-gnu/libopencv_core.so.4.2.0 to /usr/local/lib/libopencv_core.so.4.2.0 (backup original binary file to revert afterwards) like this:

/usr/lib/x86_64-linux-gnu$ ls -l libopencv_core*
-rw-r--r-- 1 root root 9124066 Feb 17  2020 libopencv_core.a
lrwxrwxrwx 1 root root      21 Feb 17  2020 libopencv_core.so -> libopencv_core.so.4.2
lrwxrwxrwx 1 root root      38 Aug 29 16:50 libopencv_core.so.4.2 -> /usr/local/lib/libopencv_core.so.4.2.0
lrwxrwxrwx 1 root root      38 Aug 29 16:51 libopencv_core.so.4.2.0 -> /usr/local/lib/libopencv_core.so.4.2.0
-rw-r--r-- 1 root root 3840856 Feb 17  2020 libopencv_core.so.4.2.0.back

Note that this would only work if you rebuild exactly the same OpenCV version than the system one to avoid any conflicts.

matlabbe commented 2 months ago

Loop closure performance comparison with CudaSift (pretty similar to OpenCV's SURF and SIFT):

introlab / rtabmap

Adding OpenCV's GPU GFTT/ OpticalFlow and CudaSift support #1330