Use my IMU and Camera,run success,but data error.ask for advice.

[guanglun]

First thanks for yours hardwork!This is a very good work!

I use my hardware:MPU6000 and mt9v034,now i can run that in /imu0 and /cam0/image_raw,run the euroc_no_extrinsic_param.launch i can get the message and locus,but the data is wrong and the locus is just go not follow i move.

there is i run log:

[ INFO] [1501068211.121514625]: position: 13.484 8.37396 -21.2694 [ INFO] [1501068211.165588810]: position: 13.4637 8.7619 -21.8698 [ INFO] [1501068211.211726307]: position: 13.4349 9.14974 -22.4898 [ INFO] [1501068211.256002523]: position: 13.3955 9.53726 -23.1311 [ INFO] [1501068211.300539934]: position: 13.3363 9.92231 -23.7912 [ INFO] [1501068211.344278917]: position: 13.2562 10.304 -24.4677 [ INFO] [1501068211.401372007]: position: 13.1909 10.6826 -25.1661 [ INFO] [1501068211.451605657]: position: 13.3228 11.0732 -25.9185 [ INFO] [1501068211.496423253]: position: 13.3618 11.4434 -26.6645 [ INFO] [1501068211.545783467]: position: 13.3926 11.8179 -27.4492 [ INFO] [1501068211.593536960]: position: 13.4314 12.1892 -28.2581 [ INFO] [1501068211.643125468]: position: 13.4453 12.5504 -29.0691 [ INFO] [1501068211.693782401]: position: 13.5169 12.8743 -29.8427 [ INFO] [1501068211.738377203]: position: 13.514 13.2774 -30.7446 [ INFO] [1501068211.783852978]: position: 13.5405 13.6432 -31.6319 [ INFO] [1501068211.829472976]: position: 13.5685 14.0061 -32.5384 [ INFO] [1501068211.879591369]: position: 13.5983 14.3674 -33.4661 [ INFO] [1501068211.922780522]: position: 13.6283 14.7285 -34.4167 [ INFO] [1501068211.975993733]: position: 13.6535 15.0895 -35.3884 [ INFO] [1501068212.027808796]: position: 13.7336 15.4495 -36.3708 [ INFO] [1501068212.075891706]: big z translation [ WARN] [1501068212.075935028]: failure detection! [ WARN] [1501068212.077594227]: system reboot! [ INFO] [1501068212.082268522]: IMU excitation not enouth! [ INFO] [1501068212.082623539]: Not enough features or parallax; Move device around [ INFO] [1501068212.084904013]: Not enough features or parallax; Move device around [ INFO] [1501068212.087540553]: Not enough features or parallax; Move device around [ WARN] [1501068212.344020025]: gyroscope bias initial calibration -1.31717 0.171048 0.929813 [ INFO] [1501068212.348381038]: misalign visual structure with IMU [ WARN] [1501068212.614799010]: gyroscope bias initial calibration 0.348552 0.308527 -0.17389 [ INFO] [1501068212.617409191]: misalign visual structure with IMU [ WARN] [1501068212.876535427]: gyroscope bias initial calibration 0.112587 -0.125255 0.00685072 [ INFO] [1501068212.879475242]: misalign visual structure with IMU [ WARN] [1501068213.113392357]: gyroscope bias initial calibration 0.0774169 -0.17866 -0.193604 [ INFO] [1501068213.116261553]: misalign visual structure with IMU [ WARN] [1501068213.311666975]: gyroscope bias initial calibration 0.169916 0.00076287 -0.0865388 [ INFO] [1501068213.315300896]: misalign visual structure with IMU W0726 19:23:33.472806 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W0726 19:23:33.479585 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W0726 19:23:33.481437 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W0726 19:23:33.483058 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. [ WARN] [1501068213.504731054]: gyroscope bias initial calibration 0.112219 0.0322218 0.000772834 [ INFO] [1501068213.509328611]: misalign visual structure with IMU W0726 19:23:33.646000 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W0726 19:23:33.647533 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W0726 19:23:33.652889 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W0726 19:23:33.659364 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. W0726 19:23:33.661514 31925 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization. [ WARN] [1501068213.681486217]: gyroscope bias initial calibration 0.103587 -0.0489546 -0.0676131 [ INFO] [1501068213.686394826]: misalign visual structure with IMU [ WARN] [1501068213.835361085]: gyroscope bias initial calibration 0.0283075 -0.0342399 -0.0378373 [ INFO] [1501068213.840587933]: misalign visual structure with IMU [ WARN] [1501068214.056249352]: gyroscope bias initial calibration 0.0427397 -0.00682596 0.00776524 [ INFO] [1501068214.062818456]: misalign visual structure with IMU [ INFO] [1501068214.065596985]: Not enough features or parallax; Move device around [ INFO] [1501068214.067585994]: Not enough features or parallax; Move device around [ INFO] [1501068214.070441184]: Not enough features or parallax; Move device around [ INFO] [1501068214.073217380]: Not enough features or parallax; Move device around [ INFO] [1501068214.076248284]: Not enough features or parallax; Move device around [ INFO] [1501068214.078329055]: Not enough features or parallax; Move device around [ INFO] [1501068214.080963211]: Not enough features or parallax; Move device around [ INFO] [1501068214.083333368]: Not enough features or parallax; Move device around [ INFO] [1501068214.087186752]: Not enough features or parallax; Move device around [ INFO] [1501068214.090016822]: Not enough features or parallax; Move device around [ INFO] [1501068214.092117697]: Not enough features or parallax; Move device around [ INFO] [1501068214.094916978]: Not enough features or parallax; Move device around [ INFO] [1501068214.097541151]: Not enough features or parallax; Move device around [ INFO] [1501068214.100137379]: Not enough features or parallax; Move device around [ INFO] [1501068214.102155446]: Not enough features or parallax; Move device around [ INFO] [1501068214.104695316]: Not enough features or parallax; Move device around [ INFO] [1501068214.106760207]: Not enough features or parallax; Move device around [ INFO] [1501068214.109655160]: Not enough features or parallax; Move device around [ INFO] [1501068214.111860823]: Not enough features or parallax; Move device around [ INFO] [1501068214.113944277]: Not enough features or parallax; Move device around [ INFO] [1501068214.116131080]: IMU excitation not enouth! [ INFO] [1501068214.116515908]: Not enough features or parallax; Move device around [ INFO] [1501068214.118969992]: IMU excitation not enouth! [ INFO] [1501068214.119278414]: Not enough features or parallax; Move device around [ INFO] [1501068214.121688079]: IMU excitation not enouth! [ INFO] [1501068214.121984015]: Not enough features or parallax; Move device around [ INFO] [1501068214.123808445]: IMU excitation not enouth! [ INFO] [1501068214.124093800]: Not enough features or parallax; Move device around [ INFO] [1501068214.147606191]: IMU excitation not enouth! [ INFO] [1501068214.147977293]: Not enough features or parallax; Move device around [ INFO] [1501068214.305161445]: IMU excitation not enouth! [ INFO] [1501068214.305575959]: Not enough features or parallax; Move device around [ INFO] [1501068214.409165335]: IMU excitation not enouth! [ INFO] [1501068214.409458436]: Not enough features or parallax; Move device around [ INFO] [1501068214.528999545]: IMU excitation not enouth! [ INFO] [1501068214.529352512]: Not enough features or parallax; Move device around [ INFO] [1501068214.649107743]: IMU excitation not enouth! [ INFO] [1501068214.649438307]: Not enough features or parallax; Move device around [ INFO] [1501068214.753049793]: IMU excitation not enouth! [ INFO] [1501068214.753380346]: Not enough features or parallax; Move device around [ INFO] [1501068214.873160605]: IMU excitation not enouth! [ INFO] [1501068214.873484567]: Not enough features or parallax; Move device around [ INFO] [1501068214.973638730]: IMU excitation not enouth! [ INFO] [1501068214.973975023]: Not enough features or parallax; Move device around [ INFO] [1501068215.093017074]: IMU excitation not enouth! [ INFO] [1501068215.093348910]: Not enough features or parallax; Move device around [ INFO] [1501068215.212917482]: IMU excitation not enouth! [ INFO] [1501068215.213221261]: Not enough features or parallax; Move device around

I just use estimate_extrinsic: 2 to calibration imu,but init success in case,the point is not control and run itself not in control.i think its becase imu data wrong,so can you tell me how to resolve it? I am sorry use the Chinese English to describe question(English so poor obvious).

Last,yours work is so 666,Thanks for your work!

[qintonguav]

Nowadays, VINS is difficult to run on your own device. In usual, the arbitrarily combined device is suffered from time unsynchronization.
We have tested on some devices, the time offsets are between 20-100ms, which is unaware in human eyes. However, for our VINS, the time period between two image is around 30ms. If the time offset between IMU and camera is larger than 30ms, which means the IMU measurements between two image is totally wrong, how can VINS work... I think most self-combined devices are suffered from this.

[maciejmatuszak]

I am using this camera+IMU, I think there is hardware synchronisation between IMU and camera. Pretty cheep for what it can do.

https://www.seeedstudio.com/Loitor-Cam2pc-Visual-Inertial-SLAM-p-2873.html

[guanglun]

@maciejmatuszak Hi,your hardware are same as mine.Are you run VINS MONO successfully?Now I think hardware synchronisation between IMU and camera is not too bad in my hardware.I try a lot way to solve the problem but without success.If you can run successfully.Can you give me some advice? Thanks!

[qintonguav]

I have bought one today. I will test it when I get this device.

[maciejmatuszak]

It runs really nice on Laptop, yet to be able to run it on TX2 see #64

[xuchen27]

My hardware is MPU9250 and GX2750 camera, and I use MPU9250 interrupt port to trigger the camera frame acquire. Beside I set a fix exposure time. At first, I set my IMU output frequency to 40 Hz and camera frame to 26 fps and got a bad performance. After that, I increase IMU frequency to 260 Hz and got a much better performance. However, MPU9250 performance is much worse than ADIS16448.

[guanglun]

@xuchen27 I am very gread to see you got good performance by MPU9250.I am wonder the reason why my hardware work so bad.Because I can't feel the locus follow my motion. Can you give me some advices about yours how to use MPU9250(MPU6000\MPU6050),how to set the acc range and gyro range.I always get the log:"big delta_angle"and"big IMU gyr bias estimation 1.74...".I think my problem is that the IMU data. Specifically, Your acc range is +-2G +-4G +-8G or +-16G? Your gyro range is +-250 +-500 +-1000 or +-2000? Are you take the data filtering? Are you do the calibration? Next is my IMU data in the still(10ms period log):

Next is my contacts,I will be glad that we can make further communicate. Mail:zhoushaopeng2016@163.com QQ:272073020

Best wishes for you!

[TAO270]

@qintony How can I test the time offsets between my IMU and Camera? Because I do not know whether the camera and IMU are precisely synchronized.

[qintonguav]

@guanglun @maciejmatuszak I have got one Loitor. I can get the stereo image, however, I cannot get the IMU. Are you modifying the SDK?

https://github.com/loitor-vis/vi_sensor_sdk

visensor_imu_have_fresh_data() always return false...

[maciejmatuszak]

I did modify it but only to allow for easy settings of exposure parameters with launch files: https://github.com/maciejmatuszak/vi_sensor_sdk check the miner-uav branch. The no imu data was usually problems with the access to /dev/ttyUSBx usually it requires user to be member of "dialout" group. I also removed the modemmamager package as it interfered with tty devices. Another source of potential problems is the device naming, the config assumes it is named ttyUSB0 but if you have more than one USB to serial adapters you will find ttyUSB1, ttyUSB2 etc. and the name depend on connection order. I tried to add symbolic link in udev rules (see the changes to install script) and specify the device by symbolic link but that did not worked, it needs more investigation. If you still have problems please post the log from launch.

[qintonguav]

@maciejmatuszak Thanks for you reply. I have already done this. The serial port open success but no IMU return. Linux 16.04 failed to get IMU data, however, my laptop which is 14.04 can get the IMU data. Anyway, I can get the IMU data now. Thanks.

[codeVerySlow]

@qintony @maciejmatuszak hi，could you please tell us the Loitor test result ,can it get a batter performance?

[qintonguav]

@guanglun @maciejmatuszak @codeVerySlow

I have tested with Loitor, it has a good performance. It can run VINS_mono well.

The time offset between IMU and camera is trivial, 3-5ms (last night is 3, today I test it is 5 ms), less than 1 IMU tic.

I run it with 30Hz left eye image and 200Hz image.

I hold the loitor, squarely walk several times.

[guanglun]

@qintony Surprise!Your result is Looks so good. Can you provide buy connection of your Loitor? And if possible can you tell me how to calibration the camera and imu?How to config the VINS mono to get good performance as yours.Can you provide The ROS node file and The project for me? Now my sensor is same as Loitor,MPU6000 and MT9V034.But I still can not get a good performance.It's so depressed. I hope to hear from you soon.

[maciejmatuszak]

I got mine here https://m.seeedstudio.com/detail/2873

[takumijash]

@qintonguav Thanks to your sharing, could you tell me how to measure the time offset between camera and IMU? I would like to check my hardware preparation. Thanks.

[yehonathanlitman]

Hey, I'm having significant issues running VINS with my loitor camera. I keep getting drifting odometry and a misalign in visual structure with IMU warning. I've calibrated the camera multiple times as pinhole and MEI (with a 0.2 pixel reprojection error) and I've tried revising the driver to see if there are any synchronization issues. I've also manually installed Eigen 3.3.3 on my system. Any ideas? I'm clueless at this point.

[yueshukun]

@xuchen27 I'm sorry to disturb you, I want to ask you some questions,I know you use mpu9250,I also use.Do you use mpu9250 with rtimulib_ros? How do you calibrate this imu. I use mpu9250 and zed to run vins-mono，but the result is really bad as the first comment.Could you tell me how about you have done. Thanks！

[Razbotics]

@qintonguav i was able to run vins mono with optor cam 2 pc nicely but the initial pose os wrong...the initial camera direction faces -y direction...can u help me with it...

[shaoxuan92]

@guanglun @maciejmatuszak @codeVerySlow

I have tested with Loitor, it has a good performance. It can run VINS_mono well.

The time offset between IMU and camera is trivial, 3-5ms (last night is 3, today I test it is 5 ms), less than 1 IMU tic.

I run it with 30Hz left eye image and 200Hz image.

I hold the loitor, squarely walk several times.

time offset between IMU and camera Hi, how to test the time offset between IMU and camera? Use td flag or test it offline? I use Mynt eye d, but it is not stable anyway. After a relatively smooth moving, it will sometimes drift far away... Have you ever tested it on Mynteyed? Thank you very much

[githubwys]

I met the same problem and solve it.

1. I uesed the camera and IMU data that are synchronized by hardware synchronize. (I will check camera and IMU data without hardware synchronize later.)
2. check the parameters. espicially the camera parameters.