/map frame is rotating randomly.

[sanjay-swain]

I am trying a simple example of handheld mapping. But when all the nodes are running I find the entire frame is constantly as shown in the video below. I have no idea what is causing the issue. I have tried checking the imu of the realsense but it is fine.

Error output:

[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:28.588) OdometryF2M.cpp:576::computeTransform() Trial with no guess succeeded!
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:28.711) OdometryF2M.cpp:566::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=83) between -1 and 275" (guess=xyz=-0.000934,-0.000062,0.000550 rpy=-0.466382,0.309671,-0.041613)
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:28.712) OdometryF2M.cpp:314::computeTransform() Failed to find a transformation with the provided guess (xyz=-0.000934,-0.000062,0.000550 rpy=-0.466382,0.309671,-0.041613), trying again without a guess.
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:28.831) OdometryF2M.cpp:576::computeTransform() Trial with no guess succeeded!
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:28.931) OdometryF2M.cpp:566::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=105) between -1 and 276" (guess=xyz=0.000442,-0.000032,0.000011 rpy=-0.416033,0.260958,0.063517)
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:28.931) OdometryF2M.cpp:314::computeTransform() Failed to find a transformation with the provided guess (xyz=0.000442,-0.000032,0.000011 rpy=-0.416033,0.260958,0.063517), trying again without a guess.
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:29.053) OdometryF2M.cpp:576::computeTransform() Trial with no guess succeeded!
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:29.155) OdometryF2M.cpp:566::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=93) between -1 and 277" (guess=xyz=-0.000225,0.000345,-0.000354 rpy=-0.344084,0.274568,-0.050905)
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:29.155) OdometryF2M.cpp:314::computeTransform() Failed to find a transformation with the provided guess (xyz=-0.000225,0.000345,-0.000354 rpy=-0.344084,0.274568,-0.050905), trying again without a guess.
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:29.271) OdometryF2M.cpp:576::computeTransform() Trial with no guess succeeded!
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:29.373) OdometryF2M.cpp:566::computeTransform() Registration failed: "Not enough inliers 0/20 (matches=87) between -1 and 278" (guess=xyz=-0.000215,-0.000215,-0.000315 rpy=-0.113920,0.309356,0.165270)
[rgbd_odometry-1] [ WARN] (2024-05-23 16:29:29.373) OdometryF2M.cpp:314::computeTransform() Failed to find a transformation with the provided guess (xyz=-0.000215,-0.000215,-0.000315 rpy=-0.113920,0.309356,0.165270), trying again without a guess.

https://github.com/introlab/rtabmap_ros/assets/59828852/769db746-1476-4148-bb72-90976dbcaab3

Any guidance will be helpful. I have tried multiple settings change from camera side but same thing is happening...

[matlabbe]

Could be a calibration issue. What are the commands you executed? I could doublecheck on my side with a realsense. The processing time looks high (200-300 ms per frame), what kind of computer are you using? Depending on the realsense you are using, I generally recommend to use the IR cameras (without ir projector) for better pose estimation, like this example https://github.com/introlab/rtabmap_ros/blob/ros2/rtabmap_examples/launch/realsense_d435i_infra.launch.py

[sanjay-swain]

I am running it on my laptop. i5 9300H processor... It is a D455. Let me look into those examples.

Edit: I have rerun the example and the same error is happening... I am recalibrating the imu.. let's see if it will fix the issue.

[matlabbe]

Can you run without the IMU? I cannot reproduce the issue with my camera and same example. Which rtabmap version are you using?

[sanjay-swain]

Yes the imu data is faulty. For some reason the realsense node is outputting bad gyro data to /camera/imu topic... But when I check on realsense-viewer the gyro and acceleration streams are completely fine... Are you using D455 camera for your tests?

I found another issue reported here https://github.com/CCNYRoboticsLab/imu_tools/issues/193. It seems like the same issue as I am facing...

I am using ros humble version rtabmap:

❯ rtabmap --version
RTAB-Map:               0.21.4
PCL:                    1.12.1
With VTK:                9.1.0
OpenCV:                  4.5.4
With OpenCV xfeatures2d: false
With OpenCV nonfree:     false
With ORB OcTree:          true
With SuperPoint Torch:   false
With Python3:            false
With FastCV:             false
With OpenGV:             false
With Madgwick:            true
With PDAL:               false
With TORO:                true
With g2o:                 true
With GTSAM:               true
With Vertigo:             true
With CVSBA:              false
With Ceres:              false
With OpenNI2:             true
With Freenect:           false
With Freenect2:          false
With K4W2:               false
With K4A:                false
With DC1394:              true
With FlyCapture2:        false
With ZED:                false
With ZED Open Capture:   false
With RealSense:          false
With RealSense SLAM:     false
With RealSense2:         false
With MYNT EYE S:         false
With DepthAI:            false
With libpointmatcher:     true
With CCCoreLib:          false
With Open3D:             false
With OctoMap:             true
With GridMap:             true
With cpu-tsdf:           false
With open chisel:        false
With Alice Vision:       false
With LOAM:               false
With FLOAM:              false
With FOVIS:              false
With Viso2:              false
With DVO:                false
With ORB_SLAM:           false
With OKVIS:              false
With MSCKF_VIO:          false
With VINS-Fusion:        false
With OpenVINS:           false

I have checked with both imu_filter_madgwick and imu_complementary_filter. It doen't look like rtabmap issue.. let me check in the respective repos

Thanks for your support!

[sanjay-swain]

Update: Well I rebuilt realsense-ros package with latest commit and it is fixed now. It was an issue with humble version of realsense-ros which was giving bad gyro data to topic...

[anath93]

@sanjay-swain same issue here in terms of drift, realsense-viewer is fine, which branch did you end up using on librealsense and realsense-ros side ?

[sanjay-swain]

Hey. I used the current ros 2 branch with the latest commits. My logic behind this issue is that ros humble version of realsense ros is outdated and librealsense library is updated to latest version. Due to this realsense Ros is not working properly.