World frame coordinate changes mid operation when IMU is involved

[thien94]

Thank you very much for making the work public.

I encounter the following issue when using the package, hope someone can help.

In Mono-Inertial or Stereo-Inertial mode, the world frame changes mid-operation (it seems to be right after InertialOptimization is completed according to the console output) as can be seen in the example below. The world frame visualization is done by adding pangolin::glDrawAxis(5.0); to Viewer.cc.

This problem is evident when I print out the Tcw value returned by the TrackMonocular function:

Translation xyz: -0.0268984828 0.147062019 0.0332303531   <--- Before InertialOptimization 
Translation xyz: -0.0243855752 0.138408974 0.033842843
build optimization graph
start optimization
end optimization
update Keyframes velocities and biases
IMU in Map 0 is initialized
Translation xyz: 0.604404151 -1.86084425 -2.65787292      <--- After InertialOptimization
Translation xyz: 0.59955281 -1.87405574 -2.67359495
...

Translation xyz: 0.418520272 -0.799674332 -3.30177617
Translation xyz: 0.412499785 -0.793865025 -3.30544758     <--- Before InertialOptimization 
start VIBA 1
build optimization graph
start optimization
end optimization
update Keyframes velocities and biases
Translation xyz: 0.407404989 -0.789338589 -3.30969024
end VIBA 1
Translation xyz: -0.0678119957 -0.580297947 -3.23924804   <--- After InertialOptimization
Translation xyz: -0.127183706 -0.560011804 -3.25348377

The issue is also somewhat unpredictable and differs from run to run. However, I don't see such behavior with Mono or Stereo mode.

I would like to pose two questions:

1. My guess is that the first world coordinate alignment corresponds to the IMU initialization (z and scale alignment), but I can't figure out why are the subsequent alignments happening?
2. Is the returned Tcw value (from TrackMonocular, TrackStereo, and TrackRGBD) recommended to be consumed by other systems that require odometry from ORB-SLAM3? If it is, the inconsistency in the coordinate frame seems undesirable. If it is not, what would be the parameter that can be used?

Thank you greatly for your time.

[richard-elvira]

When the map is initialized the world reference is set in the first keyframe. Later, IMU is initializated and the world reference is changed to aligned to it, Z axis, previously aligned with the view of the first keyframe, is aligned with respect to the gravity vector.

[thien94]

Hi @richard-elvira, thank you very much for the information.

I can understand that the world reference frame is realigned to IMU's z axis when the initialization completes, but why does the origin jump to a different location that seems to be totally random (not linked to any keyframes obtained up to that point and not consistent between runs)?

[martinakos]

Maybe related to my issue #164

[thien94]

Hi @martinakos , thank you for the info. I can confirm that the error "NOT INERTIAL LINK TO PREVIOUS FRAME!!!!" also appears in my tests, so it's possible that the two issues are one and the same.

[1216621137]

So, if I want get the real-time estimated pose, what would be the parameter that can be used? Or, it is just impossible to get the right pose in real time.

[dimaxano]

Hi, @thien94 I am reading the ORBSLAM3 paper and found this detail (on the screenshot) about the initialization scheme in ORBSLAM3. Look like a possible answer to your question

[thien94]

@dimaxano Thank you very much for the info. I didn't catch that when reading the paper.

[18851409110]

I have downloaded the code and tried to run with a mono camera but without imu information.There are some problems such as running speed is a little slow. So have you ever tried running with a real camera but not read the dataset. Maybe the dataset is the point. And have you ever tried other dataset?

BTW, is it convenient to add your wechat or other social account?

[mateosss]

but why does the origin jump to a different location that seems to be totally random (not linked to any keyframes obtained up to that point and not consistent between runs)?

Did you get an answer for this?

[AyaRabea]

I have downloaded the code and tried to run with a mono camera but without imu information.There are some problems such as running speed is a little slow. So have you ever tried running with a real camera but not read the dataset. Maybe the dataset is the point. And have you ever tried other dataset?

BTW, is it convenient to add your wechat or other social account?

I have the same problem did you find any solution?

[dantenoguera]

Hi, @thien94 I am reading the ORBSLAM3 paper and found this detail (on the screenshot) about the initialization scheme in ORBSLAM3. Look like a possible answer to your question

Didn't get what this implies. Was someone able find an explanation for the jump when IMU is involved?

[kaitaotang]

Maybe, the scale changed when the second and third initialization step. Therefore, it caused the pose of world coordinate changed as showed in your gif.

[Yeager-101]

I met the same problem, can anyone help me？