Empty IMU measurements vector!!! Not preintegrated measurement

[EugenKarasev]

Hello! Help me with my problem, please I seat with this error some days, but all solutions in reading issues didn't help with this.

I test mono-inertial ORB_SLAM3 on my rosbag with bgr images from 30 fps rolling shutter monocamera and IMU measurements with 49 Hz frequency. But every test after a few seconds fell with this error, like:

Input sensor was set to: Monocular-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Initialization of Atlas from scratch 
Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 

Camera Parameters: 
- Camera: Pinhole
- Image scale: 1
- fx: 1048.43
- fy: 1051.54
- cx: 620.221
- cy: 291.315
- k1: -0.397428
- k2: 0.177628
- p1: 0.00351412
- p2: -0.00121319
- fps: 30
- color order: BGR (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1000
- Scale Levels: 8
- Scale Factor: 1.2
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

Left camera to Imu Transform (Tbc): 
[-1.632622e-06, -0.9992404, 0.038969316, -2.9990644e-05;
 -0.23080184, -0.037916798, -0.97226173, 0.00019168481;
 0.97300076, -0.008995777, -0.23062646, -2.3621767e-05;
 0, 0, 0, 1]

IMU frequency: 49 Hz
IMU gyro noise: 0.0002 rad/s/sqrt(Hz)
IMU gyro walk: 1e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.002 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.0004 m/s^3/sqrt(Hz)
There are 1 cameras in the atlas
Camera 0 is pinhole
Starting the Viewer
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
First KF:0; Map init KF:0
New Map created with 122 points
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 167
mnInitialFrameId = 0
6 Frames set to lost
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
First KF:2; Map init KF:0
New Map created with 102 points
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 178
mnInitialFrameId = 172
14 Frames set to lost
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
First KF:5; Map init KF:2
New Map created with 237 points
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Empty IMU measurements vector!!!
Not preintegrated measurement
[ros2run]: Segmentation fault

I calibrate my camera and imu with kalibr and got these results:

Calibration results
===================
Normalized Residuals
----------------------------
Reprojection error (cam0):     mean 0.390402181164542, median 0.3190253610101713, std: 0.34872396538201356
Gyroscope error (imu0):        mean 8.638410884293534e-08, median 4.105800151162524e-10, std: 3.5084008002926186e-06
Accelerometer error (imu0):    mean 7.797051162952767e-07, median 7.962190946218488e-11, std: 3.098689588330113e-05

Residuals
----------------------------
Reprojection error (cam0) [px]:     mean 0.390402181164542, median 0.3190253610101713, std: 0.34872396538201356
Gyroscope error (imu0) [rad/s]:     mean 1.2093775238010947e-10, median 5.748120211627534e-13, std: 4.9117611204096645e-09
Accelerometer error (imu0) [m/s^2]: mean 1.0915871628133875e-08, median 1.1147067324705883e-12, std: 4.3381654236621587e-07

Transformation (cam0):
-----------------------
T_ci:  (imu0 to cam0): 
[[-0.00000163 -0.99924041  0.03896932 -0.00002999]
 [-0.23080183 -0.0379168  -0.97226171  0.00019168]
 [ 0.97300078 -0.00899578 -0.23062646 -0.00002362]
 [ 0.          0.          0.          1.        ]]

T_ic:  (cam0 to imu0): 
[[-0.00000163 -0.23080183  0.97300078  0.00006723]
 [-0.99924041 -0.0379168  -0.00899578 -0.00002291]
 [ 0.03896932 -0.97226171 -0.23062646  0.00018209]
 [ 0.          0.          0.          1.        ]]

timeshift cam0 to imu0: [s] (t_imu = t_cam + shift)
-0.06078444965185353

Gravity vector in target coords: [m/s^2]
[-0.34105986 -9.79777889  0.23585982]

Calibration configuration
=========================

cam0
-----
  Camera model: pinhole
  Focal length: [1039.5099570143711, 1042.2195539224472]
  Principal point: [618.1704778515369, 290.37109512978014]
  Distortion model: radtan
  Distortion coefficients: [-0.3935934462048003, 0.16992322720688555, 0.003903623112852698, -0.00036856266440390197]
  Type: aprilgrid
  Tags: 
    Rows: 7
    Cols: 7
    Size: 0.019 [m]
    Spacing 0.0038 [m]

IMU configuration
=================

IMU0:
 ----------------------------
  Model: calibrated
  Update rate: 49.0
  Accelerometer:
    Noise density: 0.002 
    Noise density (discrete): 0.014 
    Random walk: 0.0004
  Gyroscope:
    Noise density: 0.0002
    Noise density (discrete): 0.0014000000000000002 
    Random walk: 1e-05
  T_ib (imu0 to imu0)
    [[1. 0. 0. 0.]
     [0. 1. 0. 0.]
     [0. 0. 1. 0.]
     [0. 0. 0. 1.]]
  time offset with respect to IMU0: 0.0 [s]

If i run monocular ORB_SLAM3, I get much better results, but it's not enough for my task.

[WFram]

IMU data with 49 Hz is a low frequency itself. ORB-SLAM3 requires at least 100 Hz for IMU data stream as mentioned here. What is more important, when you have an image stream with 30 Hz, having only 49 Hz for IMU, it causes preintegration issues, because it needs more IMU readings between two consecutive frames to create a preintegrated IMU measurement. I would suggest to perform tests with reduced image stream frequency to 10 Hz and interpolated IMU data to 100 Hz in your rosbag file.

[mohamed1915]

I had the same issue and it was the frequency of the two signals. as @WFram suggested. I published the IMU at 100 and the images at 30/10 and it worked fine.

[TSoli]

Yep upsampling seemed to stop this error for me too. Unfortunately it seems that the simple linear interpolation I did to upsample might also be causing initialisation errors for the IMU. I was not able to recalibrate with Kalibr using my upsampled IMU data. To be fair I was trying to use the Tello drone and the SDK for it only exposes the IMU readings at 10Hz. I was trying to upsample to 200Hz so perhaps this was a bit ambitious.