search

UZ-SLAMLab / ORB_SLAM3

ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
GNU General Public License v3.0
6.41k stars 2.52k forks source link

Looping "IMU is not or recently initialized. Reseting active map... " #69

Open qliao001 opened 4 years ago

qliao001 commented 4 years ago

Hi many thanks for the great contribution! I have been working with monocular and multi-session version and all went well. Now I try to introduce the corresponding IMU sequence, but the system kept looping "IMU is not or recently initialized. Reseting active map... ". Could you please give some advice how can I tackle the problem? or is there any specific trick of collecting data for better IMU initialization?

Here is a screenshot, you can see that the system basically kept trying to initialize the IMU:

IMU in Map 0 is initialized PIOLKF: NOT ENOUGH EDGES Fail to track local map! IMU is not or recently initialized. Reseting active map... SYSTEM-> Reseting active map in monocular case

Screenshot from 2020-08-18 11-09-07

Thanks a lot!

ShilangChen1011 commented 4 years ago

I have the same problem. Who could explain it?

ShilangChen1011 commented 4 years ago

The reason may be that ORB-SLAM3 only supports Global Shutter cameras.

ShilangChen1011 commented 4 years ago

What is the camera type？

qliao001 commented 4 years ago

What is the camera type？

It is an Android phone embedded camera and IMU. The streaming and IMU measurements are sent by client (cell phone) and processed in the server (laptop). But I don't think it is the problem of camera, since I could run the Monocular version, but not the Monocular-Inertial version.

npcdna commented 3 years ago

yes,i meet same question. I use camera-imu parameters which calibrated by kalibr, these parameters can work well in vins, but always reset in orbslam3. when i change imu noise parameters, orbslam3 always arise segment error.

mccoychang commented 3 years ago

I also encountered this problem(these parameters can work well in vins), is there a solution?

ShreyasManjunath commented 3 years ago

I find similar error like -

New Map created with 285 points Point distribution in KeyFrame: left-> 285 --- right-> 0 build optimization graph start optimization end optimization update Keyframes velocities and biases IMU in Map 2 is initialized Fail to track local map! IMU is not or recently initialized. Reseting active map... SYSTEM-> Reseting active map in monocular case LM: Active map reset recieved LM: Active map reset, waiting... FAIL LOCAL-INERTIAL BA!!!! LM: Reseting current map in Local Mapping... LM: End reseting Local Mapping... LM: Reset free the mutex LM: Active map reset, Done!!!

This loops continously. Any one found solution?

Ehsanmb commented 3 years ago

I have the same issue! looking for solution

MiaoMiaoMeng commented 3 years ago

What is the camera type？

It is an Android phone embedded camera and IMU. The streaming and IMU measurements are sent by client (cell phone) and processed in the server (laptop). But I don't think it is the problem of camera, since I could run the Monocular version, but not the Monocular-Inertial version.

Hi, @qliao001, I met the same question, did you figure it out? I did a lot of work on camera-IMU calibrate, but it doesn't work. So I try to find the local-Inertial BA to change the threshold, but I didn't finish it yet. Do you have some idea?

MiaoMiaoMeng commented 3 years ago

Hi guys, @Ehsanmb @ShreyasManjunath @mccoychang

Today I find out how to fix this problem. If you use those APP in Android (CameraImu & MarsLogger) or other application pls make sure the coordinate is same. For me I use MarsLogger to catch info form Cam and IMU and creat bag for calibrate, and then use CameraImu to do the real time orbslam. The reason why my local-Inertial BA function was failed is that those IMU‘s coordinate is totally different.

So in this situation, you can rebuild your application to let them have a unified coordinate system.

pinkfloydfan commented 3 years ago

I've been encountering this, and believe it or not - I've 'mitigated' it by changing the noise/random walk values in the yaml file to ridiculously small numbers, like 10e-8 small. I have no idea why this works.

iamchoking commented 3 years ago

I have tried the suggestion of both @MiaoMiaoMeng and @pinkfloydfan (along with everything else on the web). But neither seem to work. I am currently using:

ZED2 (@15fps publish rate, 1280x720 resolution) ROS melodic, real-time opencv-3-2-0

topics: (published with ZED ros wrapper [zed_wrapper]) /zed2/zed_node/imu/data_raw /zed2/zed_node/left/image_rect_color /zed2/zed_node/left/image_rect_color

with imu settings calibrated from kalibr. I hope a solution to this is found soon!

*This is my first post ever on any Issue ever (i am this determined for an answer!). So if you need more info, please tell me

pinkfloydfan commented 3 years ago

I have tried the suggestion of both @MiaoMiaoMeng and @pinkfloydfan (along with everything else on the web). But neither seem to work. I am currently using:

ZED2 (@15fps publish rate, 1280x720 resolution) ROS melodic, real-time opencv-3-2-0

topics: (published with ZED ros wrapper [zed_wrapper]) /zed2/zed_node/imu/data_raw /zed2/zed_node/left/image_rect_color /zed2/zed_node/left/image_rect_color

with imu settings calibrated from kalibr. I hope a solution to this is found soon!

*This is my first post ever on any Issue ever (i am this determined for an answer!). So if you need more info, please tell me

@iamchoking sanity check: are the image frames from the camera timestamped? I was able to get ORB SLAM 3 working on an ESP32-CAM at about 12FPS, and an IMU from a drone flight controller. My workaround was setting the variances to extremely low values. DM me your configuration yaml file and I can have a look.

iamchoking commented 3 years ago

@pinkfloydfan Sure! I will post it here:

%YAML:1.0

#--------------------------------------------------------------------------------------------
# Camera Parameters ZED2 camera: HD720 mode, grab rate 15 frames per second
#--------------------------------------------------------------------------------------------
Camera.type: "PinHole"

# Camera calibration and distortion parameters (OpenCV) 
Camera.fx: 527.44
Camera.fy: 527.405
Camera.cx: 630.2
Camera.cy: 362.877

Camera.k1: -0.0416274
Camera.k2: 0.0112748
Camera.p1: -0.000148709
Camera.p2: 0.000268278

Camera.bFishEye: 0

Camera.width: 1280
Camera.height: 720

# Camera frames per second 
Camera.fps: 15

# stereo baseline times fx
Camera.bf: 63.16937904

# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1

# Close/Far threshold. Baseline times.
ThDepth: 35.0

# Transformation from camera 0 to body-frame (imu)

#found in zed_wrapper launch
#[ INFO] [1625636907.865509411]: Camera-IMU Transform: 
# 55C0C3837CD0
#0.999998 0.001782 -0.000476 -0.002000
#-0.001782 0.999998 0.000218 -0.023000
#0.000476 -0.000217 1.000000 -0.002000
#0.000000 0.000000 0.000000 1.000000

# Initial Matrix:

#   data: [ 0.999998,  0.001782, -0.000476, -0.002000,
#          -0.001782,  0.999998,  0.000218, -0.023000,
#           0.000476, -0.000217,  1.000000, -0.002000,
#           0.000000,  0.000000,  0.000000,  1.000000 ]

# BUT imu data is wrong!!!, so it is corrected by post multiplication with

#  0  0  1  0
#  -1 0  0  0
#  0 -1  0  0
#  0  0  0  1

Tbc: !!opencv-matrix
   rows: 4
   cols: 4
   dt: f
   data: [0.00725839, -0.99997021,  0.00262767,  0.02413368,
          0.00243735, -0.00261004, -0.99999362,  0.00210845,
          0.99997069,  0.00726474,  0.00241834, -0.02232050,
          0.00000000,  0.00000000,  0.00000000,  1.00000000 ]

#   data: [-1.000000, -0.001782,  0.000476, -0.002000,
#           0.001782, -1.000000, -0.000218, -0.023000,
#          -0.000476,  0.000217, -1.000000, -0.002000,
#           0.000000,  0.000000,  0.000000,  1.000000 ]
#   data: [ 1.000000,  0.001782, -0.000476, -0.002000,
#          -0.001782,  1.000000,  0.000218, -0.023000,
#           0.000476, -0.000217,  1.000000, -0.002000,
#           0.000000,  0.000000,  0.000000,  1.000000 ]

# IMU noise
#IMU.NoiseGyro: 8.6e-05 # 1.6968e-04 
#IMU.NoiseAcc: 1.4e-03 # 2.0000e-3
#IMU.GyroWalk: 2.2e-06 
#IMU.AccWalk: 8.0e-05 # 3.0000e-3
# welp I tried everything at this point
IMU.NoiseGyro: 10e-8  # 1.6968e-04 
IMU.NoiseAcc:  10e-8  # 2.0000e-3
IMU.GyroWalk:  10e-8 
IMU.AccWalk:   10e-8  # 3.0000e-3

IMU.Frequency: 400

#--------------------------------------------------------------------------------------------
# Stereo Rectification. Only if you need to pre-rectify the images.
# Camera.fx, .fy, etc must be the same as in LEFT.P
#--------------------------------------------------------------------------------------------
LEFT.height: 720
LEFT.width: 1280
LEFT.D: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data:[0.0, 0.0, 0.0, 0.0, 0.0]
LEFT.K: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [524.4577026367188, 0.0, 635.6156005859375, 0.0, 524.4577026367188, 365.53564453125, 0.0, 0.0, 1.0]
LEFT.R:  !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
LEFT.P:  !!opencv-matrix
   rows: 3
   cols: 4
   dt: d
   data: [524.4577026367188, 0.0, 635.6156005859375, -62.812259674072266, 0.0, 524.4577026367188, 365.53564453125, 0.0, 0.0, 0.0, 1.0, 0.0]

RIGHT.height: 720
RIGHT.width: 1280
RIGHT.D: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data:[0.0, 0.0, 0.0, 0.0, 0.0]
RIGHT.K: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [524.4577026367188, 0.0, 635.6156005859375, 0.0, 524.4577026367188, 365.53564453125, 0.0, 0.0, 1.0]
RIGHT.R:  !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
RIGHT.P:  !!opencv-matrix
   rows: 3
   cols: 4
   dt: d
   data: [524.4577026367188, 0.0, 635.6156005859375, -62.812259674072266, 0.0, 524.4577026367188, 365.53564453125, 0.0, 0.0, 0.0, 1.0, 0.0]

#--------------------------------------------------------------------------------------------
# ORB Parameters
#--------------------------------------------------------------------------------------------

# ORB Extractor: Number of features per image
ORBextractor.nFeatures: 1200

# ORB Extractor: Scale factor between levels in the scale pyramid   
ORBextractor.scaleFactor: 1.2

# ORB Extractor: Number of levels in the scale pyramid  
ORBextractor.nLevels: 8

# ORB Extractor: Fast threshold
# Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.
# Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST
# You can lower these values if your images have low contrast           
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7

#--------------------------------------------------------------------------------------------
# Viewer Parameters
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1
Viewer.GraphLineWidth: 0.9
Viewer.PointSize:2
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3
Viewer.ViewpointX: 0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -1.8
Viewer.ViewpointF: 500

The commented lines are the varaibles I have tried with and failed. Thanks in advance!

bboysil commented 2 years ago

Hi, I've been struggling with this error for a fewdays now. Did anyone find a solution? I was providing camera stream and IMU data from an Android, then from an iPhone. I set the IMU gyro and acc axis to the "correct" direction by playing back rosbags with recordings that work and watched carefully when the device rolled, pitched, yawed, how their IMU responded. I also tried setting the IMU walk and noise to higher and lower values, same error. Any thoughts?

I'm guessing I would really need to "calibrate" the IMU together with the camera. Can something like this help: https://github.com/ethz-asl/kalibr/wiki/camera-imu-calibration One problem is I can only send IMU data at about 60 Hz. (I tried sending it at higher rates but I would duplicate/interpolate the data).

Abner0907 commented 2 years ago

Hi guys, @Ehsanmb @ShreyasManjunath @mccoychang

Today I find out how to fix this problem. If you use those APP in Android (CameraImu & MarsLogger) or other application pls make sure the coordinate is same. For me I use MarsLogger to catch info form Cam and IMU and creat bag for calibrate, and then use CameraImu to do the real time orbslam. The reason why my local-Inertial BA function was failed is that those IMU‘s coordinate is totally different.

So in this situation, you can rebuild your application to let them have a unified coordinate system.

Hello @MiaoMiaoMeng can you share your configure file about the camera?

I have the same issue as yours, It can not detected the orb features when use imu. work good just with Mono mode. Many thanks.

maghauke commented 2 years ago

@qliao001 I believe the following can be causing this error:

Fisine commented 1 year ago

我也遇到了同样的问题，我多次标定，并且给了足够的运动，但还是循环失败，我找不出问题所在，有人解决这个问题吗

zyfff66 commented 10 months ago

I'm also having this issue, trying to recalibrate the IMU using allan_variance_ros mentioned in A summary of mono-inertial initialization issues, and suggestions for others and then adjust the IMU noise parameters:

# IMU noise (Use those from VINS-mono)
IMU.NoiseGyro: 1e-3 # rad/s^0.5
IMU.NoiseAcc: 1e-2   # m/s^1.5
IMU.GyroWalk: 1e-7 # rad/s^1.5
IMU.AccWalk: 1e-7 # m/s^2.5
IMU.Frequency: 200

Then things got better.

MaZhaoYong commented 3 months ago

我也遇到了同样的问题，我多次标定，并且给了足够的运动，但还是循环失败，我找不出问题所在，有人解决这个问题吗

hello，我也遇到了同样的问题，请问您解决了么，麻烦不吝赐教了