search

HKUST-Aerial-Robotics / VINS-Mono

A Robust and Versatile Monocular Visual-Inertial State Estimator
GNU General Public License v3.0
5.04k stars 2.1k forks source link

Misalign of visual structure with IMU, Pose upside down, translations not working #333

Open maximpavliv opened 4 years ago

maximpavliv commented 4 years ago

Hello,

I want to use VINS-Mono on my eye-tracking device, but I get a very weird behavior.

My case is the following : My eye tracking device has a rolling shutter camera running at 25Hz, software-synchronized with an IMU running at 90Hz. I know that the "rolling shutter camera - not hardware-synchronized system" is the worst scenario, but I believe that it should still work at least a little bit when doing slow movements (slow movements tackling the potential unsychronization issues).

I followed precisely the setup instructions, and when I launch VINS-Mono with data from my device, VINS-Mono restarts initialization several times (throwing the "misalign visual structure with IMU", the "big IMU acc bias estimation x", and the "_W0311 14:01:45.245322 18047 levenberg_marquardtstrategy.cc:116] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization" messages). It eventually successfully starts, but the marker is upside down (thus meaning that x and y rotations are inverted). The program doesn't react to translational movements, and sometimes the marker just randomly flies away until infinity. The rotation movements do work, but since the marker is upside down, rotations around x and y axes are inverted.

This behavior is systematic, even with slow movements, so I would doubt that it is a synchronization problem. The fact that the pose starts always upside down and the "misalign" message make me think that it is an external calibration issue, but I did the calibration many many times using Kalibr, and got a similar result each time, and even the "no-extrinsic-params" mode gives the same extrinsic parameters, and the same behavior.

The tracking of the feature points works well, so I don't think this is an issue either.

Do you have any idea about what I might be doing wrong? Is it all due to the hardware I am using?

shaozu commented 4 years ago

Hi, firstly I suggest you to check your IMU stream, e.g. whether the IMU reading has a large bias. You can do integration and double-integration on IMU readings, the trajectory should be normal for seconds.

SerpentWindy commented 6 months ago

Hi, firstly I suggest you to check your IMU stream, e.g. whether the IMU reading has a large bias. You can do integration and double-integration on IMU readings, the trajectory should be normal for seconds.

Excuse me, although it has been a long time, I would like to ask how I should correct the IMU data if the trajectory obtained by double integration with IMU data goes wrong within a few seconds.