search

introlab / rtabmap

RTAB-Map library and standalone application
https://introlab.github.io/rtabmap
Other
2.84k stars 786 forks source link

When using Azure Kinect for house scanning, tracking often fails #952

Closed zhangzscn closed 1 year ago

zhangzscn commented 1 year ago

When using the iPhone's RTABMap app, house scanning on the iPhone13 Pro Max looked good. But using the windows version of RTABMap, house scanning with Azure Kinect, often failed to track. What's the difference between the RTABMap apple app and RTABMap windows? Results in a large gap between the two software effects.

Software: RTABMap-0.20.16-win64-cuda11-1.exe Configuration:config_k4a_icp.ini

matlabbe commented 1 year ago

The icp config you are using assumes that you scan objects/room with enough geometry constraints. ICP won't be robust when losing tracking. You may use RGB-D odometry instead (set default parameters, then select Kinect Azure from Detection->Select source menu).

The big difference between iOS and the standalone is the odometry used. For the standalone, it is RGB-D or ICP odometry (from this repo). For iOS, it is a proprietary visual inertial odometry (VIO) approach (ARKit). It is possible to use third party VIO approaches in rtabmap standalone, but they are usually working only in linux. For example, here are the other odometry approaches that can be used in rtabmap standalone with the right build config (mostly only on linux / ROS):

--  Odometry Approaches:
--   With loam_velodyne        = NO (WITH_LOAM=OFF)
--   With floam                = NO (WITH_FLOAM=OFF)
--   With libfovis             = NO (WITH_FOVIS=OFF)
--   With libviso2             = NO (WITH_VISO2=OFF)
--   With dvo_core             = NO (WITH_DVO=OFF)
--   With okvis                = NO (WITH_OKVIS=OFF)
--   With msckf_vio            = NO (WITH_MSCKF_VIO=OFF)
--   With VINS-Fusion          = NO (WITH_VINS=OFF)
--   With OpenVINS             = NO (WITH_OPENVINS=OFF)
--   With ORB_SLAM             = NO (WITH_ORB_SLAM=OFF)

See this paper for a comparison of some of them (e.g., Table 7 for VIO). As some of them are EOL, I don't keep supporting them with new rtabmap releases. I often refer to this docker file to see how I was able to build them with rtabmap in the past.

Nevertheless, ARkit's VIO is pretty robust and accurate in general (similar to Google Tango back in the days), though sometimes it can jump 1 meter for no reason when the camera is looking at textureless walls/ceiling. To have similar results with Kinect Azure, we would need to release rtabmap with a VIO approach, maybe OpenVINS as it is still supported and if it can be built on windows (though is seems pretty much ROS2 dependent).

zhangzscn commented 1 year ago

thanks