search

HKUST-Aerial-Robotics / VINS-Fusion

An optimization-based multi-sensor state estimator
GNU General Public License v3.0
3.46k stars 1.38k forks source link

VINS-FUSION drift #41

Open Lsmartworld opened 5 years ago

Lsmartworld commented 5 years ago

My devices are MYNTEYED and tx2 .During the process of using VINS-FUSION, I found that when running stereo + IMU, the drift is very powerful. But when running stereo, it is very stable. Would you like to ask if you have encountered such a problem?

lishanggui commented 4 years ago

I have the same problem as you! do you slove the problem? @Lsmartworld

iliis commented 4 years ago

VINS is requires accurate timing. IMU und camera frames need to be synchronized well - in the order of a millisecond or better, so hardware synchronization is essential. If your timestamps are off by even a single frame or have high jitter (for example because you're measuring them on the PC using ros::Time::now()) VINS will diverge quickly. While the pipline tries to estimate a constant time offset between IMU and camera[s], this doesn't work too well and is more of a crutch than a real fix for wrong or bad synchronization.

lishanggui commented 4 years ago

@iliis Thank you very much for your answers. before running stereo+imu, I used the kalibr program to do stereo calibration and imu-camera calibration for mynteye camera. The calibration results should look pretty good (as shown in the link) ). When running stero, it is more stable than imu+stereo. How can I achieve better synchronization?r is the problem with the hardware device I am using that the time synchronization is not done well? report-imucam-mynteyemynt_imu_camera_calib_202006030038.pdf report-cam-mynteye_camera_calib_202006030038.pdf

daoran commented 4 years ago

@lishanggui Your calibration result is pretty good. You can try increasing your IMU params inside config file. Like the following

acc_n: 0.1          
gyr_n: 0.01          
acc_w: 0.001         
gyr_w: 0.0001
iliis commented 4 years ago

The calibration looks indeed good (although it doesn't seem to cover the full frame). And as this is a commercial camera the time synchronization should be fine as well, I would assume. So maybe the problem is something else. Did you double check all the transformations? Maybe some axis or tranformation got flipped or inverted.

lishanggui commented 4 years ago

@lishanggui Your calibration result is pretty good. You can try increasing your IMU params inside config file. Like the following

acc_n: 0.1          
gyr_n: 0.01          
acc_w: 0.001         
gyr_w: 0.0001

Thank you very much！Imu's noise is calibrated by imu_utils. According to the appendix of this article（J. L. Crassidis, “Sigma-point Kalman filtering for integrated GPS and inertial navigation,” IEEE Trans. Aerosp. Electron. Syst., vol. 42, no. 2, pp. 750–756, Apr. 2006.）, the calibration results need to be converted into discrete values, imu_utils results: update_rate: 200.0 #Hz (for discretization of the val)

Accelerometers

accelerometer_noise_density: 1.0281611399787572e-02 #Noise density (continuous-time) accelerometer_random_walk: 2.1568297052184193e-04 #random walk

Gyroscopes

gyroscope_noise_density: 2.9057508704048753e-03 #Noise density (continuous-time) gyroscope_random_walk: 8.4607958318476167e-05 #random walk

so, acc_n: 1.0281611399787572e-02*200^0.5 =0.145403943

gyr_n: 2.9057508704048753e-03*200^0.5=0.041093523 acc_w: 2.1568297052184193e-04/(200^0.5)=0.000015251 gyr_w:8.4607958318476167e-05 / (200^0.5)=0.000005983

At the same time, I also changed multiple sets of these parameters for experiments, including the values you provided, but the results have not improved！

lishanggui commented 4 years ago

@iliis Thank you very much! I have checked the entire vins-fusion program and my configuration. I have not found where the problem is？And I'm sorry I don't understand where to check "Maybe some axis or tranformation got flipped or inverted". This is my entire config configuration file. config.tar.gz

daoran commented 4 years ago

@lishanggui I see your config files. When you use Kalibr to calibrate your camera, you should use pinhole-equi camera model.

lishanggui commented 4 years ago

@daoran I used the pinhole-equi model to calibrate the camera(as shown in the link)，and write into the configuration file. During the process of using VINS-FUSION, it also stereo better than stereo+imu. According to mynteye camera official documents, the use of pinhole-equi or omni-radtan models will not affect the results of vins-fusion (https://mynt-eye-s-sdk.readthedocs.io/zh_CN/2.3.6/src/slam/how_to_use_kalibr.html). calibration result(pinhole-equi model): report-cam-mynteyemynt_imu_camera_calib_202006030028_pinhole-equi.pdf report-imucam-mynteyemynt_imu_camera_calib_202006030038_pinhole-equi.pdf config(pinhole-equi model): mynt_s1030_pinhole_equi_config.tar.gz stereo result: rviz_screenshot_2020_06_10-21_27_42 stereo+imu result: rviz_screenshot_2020_06_10-21_19_31

daoran commented 4 years ago

@lishanggui I don't know. It's weird, have you tried turning off estimate_extrinsic?

lishanggui commented 4 years ago

@daoran @iliis Yes, I tried, but the results did not improve, or worse. Have you done this experiment comparison? How are your stereo+imu results? What sensor did you use for the experiment？thank you for your help.

daoran commented 4 years ago

@lishanggui I used 1030 and 2110 before, the results got from vo and vio were close. We can talk by email.

lishanggui commented 4 years ago

@daoran thank you！ I have sent email to you. looking forward to your reply！

getupcurry commented 4 years ago

@lishanggui ,Hi,when you run VINS-Fusion camera+imu with mynteye, do you have wait for imu... about 4-5 times each time an image is processed? like:

rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/mynteye/mynteye_stereo_imu_config.yaml
[ INFO] [1598978007.569093781]: init begins
config_file: /home/yu/catkin_ws/src/VINS-Fusion/config/mynteye/mynteye_stereo_imu_config.yaml
USE_IMU: 1
IMU_TOPIC: /mynteye/imu/data_raw
result path /home/tong/output//vio.csv
[ WARN] [1598978007.573738225]:  Optimize extrinsic param around initial guess!
camera number 2
[ INFO] [1598978007.706211535]: Synchronized sensors, fix time offset: 0
[ INFO] [1598978007.706234998]: ROW: 480 COL: 752 
 exitrinsic cam 0
 0.00428124    -0.99997  -0.0064529
   0.999969  0.00423843  0.00663446
-0.00660691  -0.0064811    0.999957
5.25834e-05  -0.0421747   0.0192387
 exitrinsic cam 1
 0.00165252   -0.999985 -0.00520307
   0.999949  0.00160076  0.00993615
-0.00992768 -0.00521922    0.999937
-0.00292519   0.0743791   0.0193027
set g   0   0 9.8
[ INFO] [1598978007.706408891]: reading paramerter of camera /home/mathieu/catkin_ws/src/VINS-Fusion/config/mynteye/left_mei.yaml
[ INFO] [1598978007.706540475]: reading paramerter of camera /home/mathieu/catkin_ws/src/VINS-Fusion/config/mynteye/right_mei.yaml
MULTIPLE_THREAD is 1
[ WARN] [1598978007.706631145]: waiting for image and imu...
init first imu pose
averge acc 9.505009 -1.207654 -1.477698
init R0 
 0.196851  0.620425  0.759159
        0  -0.77431  0.632807
 0.980433 -0.124568 -0.152423
wait for imu ... 
[ WARN] [1598978009.177034355]: gyroscope bias initial calibration -0.000124802 -2.62956e-05 -0.000217221
[ INFO] [1598978009.224925882]: Initialization finish!
time: 1598978009.169869, t: -0.000665 -0.000257 -0.002229 q: -0.260500 0.728589 0.213583 0.596390 
time: 1598978009.249849, t: -0.001597 -0.000752 -0.004902 q: -0.260719 0.728493 0.213739 0.596355 
time: 1598978009.329839, t: -0.001694 -0.000810 -0.005014 q: -0.260702 0.728537 0.213725 0.596313 
time: 1598978009.409819, t: -0.001755 -0.000876 -0.005074 q: -0.260691 0.728578 0.213717 0.596271 
time: 1598978009.489799, t: -0.001771 -0.000903 -0.005098 q: -0.260679 0.728619 0.213709 0.596230 
time: 1598978009.569779, t: -0.002233 0.001027 -0.006803 q: -0.261969 0.728236 0.214826 0.595730 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
time: 1598978009.649769, t: 0.000186 -0.000272 -0.003639 q: -0.264328 0.726812 0.217166 0.595578 
time: 1598978009.729749, t: 0.000196 -0.001327 -0.002751 q: -0.266952 0.725523 0.219574 0.595097 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
time: 1598978009.809729, t: -0.003321 -0.005946 -0.003968 q: -0.255608 0.731011 0.209377 0.597033 
time: 1598978009.889709, t: 0.009192 0.011846 -0.002178 q: -0.264213 0.726654 0.217948 0.595536 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ... 
time: 1598978009.969699, t: 0.009783 0.012216 0.004889 q: -0.264305 0.726750 0.217985 0.595365 
time: 1598978010.049679, t: 0.008858 0.014150 0.011544 q: -0.264964 0.726927 0.218214 0.594772 
time: 1598978010.129659, t: 0.010310 0.013431 0.015126 q: -0.263210 0.727664 0.216651 0.595220 
time: 1598978010.209639, t: 0.012167 0.016279 0.016885 q: -0.263577 0.727451 0.216909 0.595225 
wait for imu ... 
wait for imu ... 
wait for imu ... 
wait for imu ...
Shengnan-Zhu commented 2 years ago

@getupcurry Hi, I got the same problem "wait for imu", have you solved it?

bulingda commented 1 year ago

Hello,have you solved it? @Lsmartworld @lishanggui @Shengnan-Zhu

ToooJOHN commented 6 months ago

Hello,have you solved it? @Lsmartworld @lishanggui @Shengnan-Zhu

ashwinjoseph95 commented 2 months ago

Hey any updates on the issue about "wait for imu ... " ?