search

thien94 / orb_slam3_ros

A ROS implementation of ORB_SLAM3
GNU General Public License v3.0
261 stars 81 forks source link

Which topic should I subscribe to if I want to output real-time location information of the entire system? #13

Closed Shang-cl closed 1 year ago

Shang-cl commented 1 year ago

I am now using a RealSense D455 camera for testing, and the node used is Stereo_Inertial_ros. I would like to know which topic should I subscribe to if I need to view real-time positioning information?

thien94 commented 1 year ago

Hi @Shang-cl , the /orb_slam3/camera_pose and /orb_slam3/body_odom are the two topics that you can subscribe to for the real-time pose data.

Shang-cl commented 1 year ago

@thien94 Hello, thank you very much for your timely reply. Is the pose information corresponding to these two topics the camera and the IMU? Which one should be used in practical engineering applications? During the actual test process, I found that there is a big problem with my positioning information. I moved forward about 10m with a certain point as the origin, but the corresponding movement distance obtained from the positioning information output is only about 3m, which is about the actual movement distance 1/3, I carefully searched for any possible problems, but I couldn't solve this problem well. Do you have any good suggestions or solutions? Thanks again!

thien94 commented 1 year ago

Is the pose information corresponding to these two topics the camera and the IMU? Which one should be used in practical engineering applications?

Yes, that is correct. The topic you should use depends on your requirements, but generally the IMU-based body odom is preferred. Do note that because of the way ORB-SLAM3 works, the output rate of the body odometry is the same or lower than the camera framerate (20-30Hz), and not the same as the IMU frequency (200Hz).

During the actual test process, I found that there is a big problem with my positioning information. ... I moved forward about 10m with a certain point as the origin ...

This might be a scale initialization issue, whereby you need to move the camera with some movements in all 3D axes for the system to correctly estimate the scale. There are some research works that try to address this issue without random motion such as https://dl.acm.org/doi/10.1145/3469213.3470677, but they are certainly not included of this repo.

For now, the easiest solution is to move the camera randomly in all directions at the start of each experiment, similar to the motion in this video https://youtu.be/O04EywptgH8

Shang-cl commented 1 year ago

@thien94 I am using RealSense D455, and I have made sure that I have fully calibrated the relevant parameters, and I have fully encouraged the IMU to complete the initialization before performing slam. Interestingly, I have done the following tests: I installed the camera on the side of the car body, which means that when the car is moving forward, the image captured by the camera will be of the side of the car body. In this case, I found that the positioning is very accurate. So I have the following ideas:

  1. When the direction of the camera's viewing angle is consistent with the direction of movement of the body, since the scene in front of it changes little, and the feature points in the image hardly change, then there may be a problem of incorrect scale, so I guess that if you filter out the far Feature points that do not change over time may help.
  2. The calibration of the camera and IMU is not accurate. But this problem was quickly ruled out by me. So based on the above analysis, I think it is probably the first reason that caused my problem. Of course, I am just guessing (because I am a novice in the direction of visual SLAM). I don't know if my analysis is correct, thank you for your patience in answering the questions I encountered!
thien94 commented 1 year ago

Hi @Shang-cl , thanks for your detailed analysis. I believe you are on the right track to figuring out the exact issue.

It is indeed the case that the moving direction of the camera has some effects on the error. In fact, pure forward motion is one of the degenerate cases for the visual odometry problem (a way of saying that in general, there can be multiple plausible solutions for the camera poses). You can read more on this and other cases here https://www.researchgate.net/publication/258224859_Scalable_Visual_Navigation_for_Micro_Aerial_Vehicles_using_Geometric_Prior_Knowledge/figures?lo=1

Have you run the algorithm with the D455 in stereo or stereo-inertial modes? With 2 cameras, the issue might be mitigated somewhat. However, it still depends on the scene: if everything is far away from both cameras then the left and right images will have no difference, and the core issue remains. Nonetheless, it is still worth a try.

Shang-cl commented 1 year ago

@thien94 Thanks again for your reply. I have already used the RealSense D455 binocular camera to run the algorithm, but after your suggestion and my own thinking, I think the current problem is largely a problem of the application scenario. Your 'pure direction Pre-motion is one of the degenerate cases of the visual odometry problem' This suggestion helped me a lot, something I didn't know. I think your analysis and suggestions can help me solve the problems I encountered. I will close this question. Thank you again for your patience in answering my doubts!