Open emilymes opened 1 year ago
matlabbe
commented
1 year ago Hi, I think you meant --Vis/FeatureType 0, but you should build rtabmap_ros with your rtabmap built from source. To make sure the right one is detected on catkin make, remove the ros binaries:
sudo apt remove ros-noetic-rtabmap*cheers, Mathieu
emilymes
commented
1 year ago Yes, thank you! The binaries were not installed, but in the end a complete rebuild worked. Now I am able to roslaunch rtabmap_ros using all the feature types! Except for superpoint..
roslaunch rtabmap_ros rtabmap.launch args:="--delete_db_on_start --Vis/FeatureType 11 --SuperPoint/ModelPath ~/catkin_ws/superpoint.pt" gives the following errors:
Following [https://github.com/introlab/rtabmap_ros/issues/28](the debugging instructions) this is the gdb backtrace for both died processes:
I suspect it maybe goes wrong in the rtabmap_ros/rtabmap_odom/src/RGBDOdometryNode.cpp and rtabmap_ros/rtabmap_slam/src/CoreNode.cpp, and then specifically when loading the nodelet nodelet.load(nodelet_name, "rtabmap_slam/rtabmap", remap, nargv);
Would you have an idea how it can be fixed to get SuperPoint working? Thank you in advance.
emilymes
commented
1 year ago Hi Mathieu, I have spent some time debugging and printing statements to see where it goes wrong, and this is what I found:
nodelet.load() line, but does not exit it. Once in nodelet.load() it goes into the OnInit in src/CoreWrapper.cpp, the LoadParameters, the SaveParameters, and then it goes in the ~CoreWrapper function, which is also exits without errors. By then I can not determine which function it enters. I thought that once the ~CoreWrapper function exits, it will go back to the ros::spin() line in CoreNode.cpp but this is not the case.nodelet.load(), but this time it goes into the OnInit of the OdometryROS.cpp, updates the parameters, and is then stuck on `odometry = Odometry::create(parameters_);` But I don't know where exactly in that line it goes wrong.Do you have any idea what the problem could be? Thanks in advance.
emilymes
commented
1 year ago Hi Mathieu,
I have continued with my debugging and printing, and have now narrowed it down to one line, namely:
torch::load(model_, uReplaceChar(path, '~', UDirectory::homeDir())); in the SPDetector::SPDetector() function in the SuperPoint.cc file. The path should be correct and I have tried both superpoint.pt and superpoint_v1.pth files.
I don't get an error message on that line, but my print statement after that line does not print, and instead the destructor Odometry::~OdometryROS() function is executed. This suggests that an error occurred during the loading process.
Would you have any idea what causes this issue? Have you seen this problem?
I am on Ubuntu 20.04 and use libtorch-cxx11-abi-shared-with-deps-2.0.1+cpu (but get the same issue when using with libtorch2.0.1+cuda11.7 on another PC with a GPU)
rtabmap --version gives:
RTAB-Map: 0.21.1
PCL: 1.10.0
With VTK: 7.1.1
OpenCV: 4.2.0
With OpenCV xfeatures2d: true
With OpenCV nonfree: true
With ORB OcTree: true
With SuperPoint Torch: true
With Python3: trueand cmake .. in ~/rtabmap/build gives:
-- MOBILE_BUILD=OFF
-- Eigen found (include: /usr/include/eigen3, version: 3.3.7)
-- OpenNI found (include: /usr/include/ni, lib: /usr/lib/libOpenNI.so)
-- The imported target "vtkParseOGLExt" references the file
"/usr/bin/vtkParseOGLExt-7.1"
but this file does not exist. Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
"/usr/lib/cmake/vtk-7.1/VTKTargets.cmake"
but not all the files it references.
-- The imported target "vtkRenderingPythonTkWidgets" references the file
"/usr/lib/x86_64-linux-gnu/libvtkRenderingPythonTkWidgets.so"
but this file does not exist. Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
"/usr/lib/cmake/vtk-7.1/VTKTargets.cmake"
but not all the files it references.
-- The imported target "vtk" references the file
"/usr/bin/vtk"
but this file does not exist. Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
"/usr/lib/cmake/vtk-7.1/VTKTargets.cmake"
but not all the files it references.
-- The imported target "pvtk" references the file
"/usr/bin/pvtk"
but this file does not exist. Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
"/usr/lib/cmake/vtk-7.1/VTKTargets.cmake"
but not all the files it references.
-- QHULL found (include: /usr/include, lib: optimized;/usr/lib/x86_64-linux-gnu/libqhull.so;debug;/usr/lib/x86_64-linux-gnu/libqhull.so)
-- OpenNI found (include: /usr/include/ni, lib: /usr/lib/libOpenNI.so)
-- PCL definitions don't contain "-march=native", make sure all libraries using Eigen are also compiled without that flag to avoid some segmentation faults (with gdb referring to some Eigen functions).
-- Found Sqlite3: /usr/include /usr/lib/x86_64-linux-gnu/libsqlite3.so
-- Found OpenMP: /usr/lib/gcc/x86_64-linux-gnu/10/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so
-- Add PCL_OMP to definitions
-- Found OpenCV: /usr/local/include/opencv4
-- Found PCL: /usr/include/pcl-1.10;/usr/include/eigen3;/usr/include;/usr/include/ni;/usr/include/vtk-7.1;/usr/include/freetype2;/usr/include/x86_64-linux-gnu
-- Found ZLIB: /usr/include
-- VTK_RENDERING_BACKEND=OpenGL2
-- Found Torch: /home/emily/libtorch-cxx11-abi-shared-with-deps-2.0.1+cpu/libtorch/include;/home/emily/libtorch-cxx11-abi-shared-with-deps-2.0.1+cpu/libtorch/include/torch/csrc/api/include
-- Found Python3
-- Found Freenect: /usr/include
-- Found OpenNI2: /usr/include/openni2
-- Found DC1394: /usr/include/dc1394
-- Found g2o (targets)
-- g2o include dir: /opt/ros/noetic/include
-- Old g2o factory version detected without shared ptr (factory file: /opt/ros/noetic/include/g2o/core/factory.h).
-- GTSAM include directory: /usr/lib/cmake/GTSAM/../../../include
-- FlyCapture2_INCLUDE_DIR=FlyCapture2_INCLUDE_DIR-NOTFOUND
-- FlyCapture2_LIBRARY=FlyCapture2_LIBRARY-NOTFOUND
-- Triclops_INCLUDE_DIR=Triclops_INCLUDE_DIR-NOTFOUND
-- Triclops_LIBRARY=Triclops_LIBRARY-NOTFOUND
-- FlyCaptureBridge_LIBRARY=FlyCaptureBridge_LIBRARY-NOTFOUND
-- Found Boost: /usr/include (found version "1.71.0") found components: thread filesystem system program_options date_time chrono atomic
-- Found libpointmatcher: /opt/ros/noetic/include;/usr/include/eigen3
-- Found Boost: /usr/include (found version "1.71.0") found components: thread filesystem system program_options date_time chrono timer serialization atomic
-- Found octomap 1.9.8: /opt/ros/noetic/include
-- Architecture: x86_64
-- Found Pthreads
-- --------------------------------------------
-- Info :
-- RTAB-Map Version = 0.21.1
-- CMAKE_VERSION = 3.16.3
-- CMAKE_INSTALL_PREFIX = /home/emily/catkin_ws/devel
-- CMAKE_BUILD_TYPE = Release
-- CMAKE_INSTALL_LIBDIR = lib
-- BUILD_APP = ON
-- BUILD_TOOLS = ON
-- BUILD_EXAMPLES = ON
-- BUILD_SHARED_LIBS = ON
-- CMAKE_CXX_FLAGS = -fmessage-length=0 -fopenmp -std=c++14
-- FLANN_KDTREE_MEM_OPT = OFF
-- PCL_DEFINITIONS = -DDISABLE_OPENNI2;-DDISABLE_PCAP;-DDISABLE_PNG;-DDISABLE_LIBUSB_1_0
-- PCL_VERSION = 1.10.0
--
-- Optional dependencies ('*' affects some default parameters) :
-- *With OpenCV 4.2.0 xfeatures2d = YES, nonfree = YES (License: Non commercial)
-- With Qt 5.12.8 = YES (License: Open Source or Commercial)
-- With VTK 7.1 = YES (License: BSD)
-- With external SQLite3 = YES (License: Public Domain)
-- With ORB OcTree = YES (License: GPLv3)
-- With SupertPoint = YES (License: GPLv3) libtorch=2.0.1
-- With Python3.8 = YES (License: PSF)
-- With Madgwick = YES (License: GPL)
-- With FastCV = NO (FastCV not found)
-- With PDAL = NO (PDAL not found)
--
-- Solvers:
-- With TORO = YES (License: Creative Commons [Attribution-NonCommercial-ShareAlike])
-- *With g2o 1.0.0 = YES (License: BSD)
-- *With GTSAM 4.0.3 = YES (License: BSD)
-- *With Ceres = NO (WITH_CERES=OFF)
-- With MRPT = NO (MRPT not found)
-- With VERTIGO = YES (License: GPLv3)
-- With cvsba = NO (WITH_CVSBA=OFF)
-- *With libpointmatcher 1.3.1 = YES (License: BSD)
-- With CCCoreLib = NO (WITH_CCCORELIB=OFF)
-- With Open3D = NO (WITH_OPEN3D=OFF)
-- With OpenGV = NO (OpenGV not found)
--
-- Reconstruction Approaches:
-- With OCTOMAP 1.9.8 = YES (License: BSD)
-- With CPUTSDF = NO (WITH_CPUTSDF=OFF)
-- With OpenChisel = NO (WITH_OPENCHISEL=OFF)
-- With AliceVision = NO (WITH_ALICE_VISION=OFF)
--
-- Camera Drivers:
-- With Freenect = YES (License: Apache v2 and/or GPLv2)
-- With OpenNI2 = YES (License: Apache v2)
-- With Freenect2 = NO (libfreenect2 not found)
-- With Kinect for Windows 2 = NO (Kinect for Windows 2 SDK not found)
-- With Kinect for Azure = NO (Kinect for Azure SDK not found)
-- With dc1394 = YES (License: LGPL)
-- With FlyCapture2/Triclops = NO (Point Grey SDK not found)
-- With ZED = NO (ZED sdk and/or cuda not found)
-- With ZEDOC = NO (ZED Open Capture not found)
-- With RealSense = NO (librealsense not found)
-- With RealSense2 = NO (librealsense2 not found)
-- With MyntEyeS = NO (mynteye s sdk not found)
-- With DepthAI = NO (WITH_DEPTHAI=OFF)
--
-- Odometry Approaches:
-- With loam_velodyne = NO (WITH_LOAM=OFF)
-- With floam = NO (WITH_FLOAM=OFF)
-- With libfovis = NO (WITH_FOVIS=OFF)
-- With libviso2 = NO (WITH_VISO2=OFF)
-- With dvo_core = NO (WITH_DVO=OFF)
-- With okvis = NO (WITH_OKVIS=OFF)
-- With msckf_vio = NO (WITH_MSCKF_VIO=OFF)
-- With VINS-Fusion = NO (WITH_VINS=OFF)
-- With OpenVINS = NO (WITH_OPENVINS=OFF)
-- With ORB_SLAM = NO (WITH_ORB_SLAM=OFF)
-- Show all options with: cmake -LA | grep WITH_
-- --------------------------------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /home/emily/rtabmap/buildThank you in advance!
Hi, I am trying to use the nonfree opencv algorithms in combination with rtabmap_ros, to compare the performance of the various visual features. I have built opencv from source with nonfree algorithms enabled and was able to build rtabmap from source as well, which works as expected.
rtabmap --versiongivesand when I use the rtabmap standalone I can see in Window > Preferences > Visual Registration > Visual Feature that all the feature detectors are available.
However, when I
roslaunch rtabmap_ros rtabmap.launch args:="--Vis/FeatureType SURF"for example, I get the following warning:[ WARN] (2023-04-24 16:32:08.975) Features2d.cpp:529::create() SURF and SIFT features cannot be used because OpenCV was not built with xfeatures2d module. GFTT/ORB is used instead.Also in Window > Preferences > Visual Registration the nonfree feature detectors are not available.Would you happen to know what goes wrong? Thank you in advance!