State estimation divergence on performing with ZED2

[Aishkrish18]

Hey, First of all thank you so much for this great open source work and for your instant replies for clearing our doubts. I am performing state estimation for ZED2 camera and I am currently facing this problem where the tracking goes off the rviz grid and it shoots off. I perfomed correct calibration and i have pasted them here. I am not sure if there are certain specific parameters that i have to tune for having them exactly follw the path of the camera.

I have herewith attavched the calibration files and the screenshot of how it shoots off from the grid. zed2_cam_imu_calib4-results-imucam.txt

[Aishkrish18]

Update: This action was somewhat similar for d435 as well. It would be great if someone can assist me on how to fix it, so that i can compare the results and make a conclusion. i plan to use this algorithm for my master thesis to work and tune more on top of the existing algorithm :)

[goldbattle]

Have you tried calibrating the IMU intrinsics with kalibr and also providing those?

On Tue, Aug 1, 2023, 7:02 AM Tetrix @.***> wrote:

Update: This action was somewhat similar for d435 as well. It would be great if someone can assist me on how to fix it, so that i can compare the results and make a conclusion. i plan to use this algorithm for my master thesis to work and tune more on top of the existing algorithm :)

— Reply to this email directly, view it on GitHub https://github.com/rpng/open_vins/issues/366#issuecomment-1660399089, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ6TYR2235OI3J2FSXKDY3XTEEBFANCNFSM6AAAAAA254LIVY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Aishkrish18]

By imu intrinsics you mean the allan variance ros -imu bias values or imu-camera calibration? Or am i missing something other than that?
These are D435 imu cam calibration parameters- below.

[goldbattle]

Kalibr IMU-camera calibration, should be the --imu-models parameter to scale-missalignment I believe? Allan variance will get your noise parameters, not the intrinsics.

On Tue, Aug 1, 2023 at 8:47 AM Tetrix @.***> wrote:

By imu intrinsics you mean the allan variance ros -imu bias values or imu-camera calibration? Or am i missing something other than that?

— Reply to this email directly, view it on GitHub https://github.com/rpng/open_vins/issues/366#issuecomment-1660594958, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ6TYXCGRX3WRZA4T2BOT3XTEQKDANCNFSM6AAAAAA254LIVY . You are receiving this because you commented.Message ID: @.***>

[Aishkrish18]

Not sure if i understood you correctly, but you mean I should calibrate imu camera using the model parameter scale-misalignment instead of "calibrated" is it? for ZED2 and d435? This will help solving the issue?

[Aishkrish18]

On performing calibration once again by using scale-misalignment, i was able to retrieve the intrinsics of IMU. But this is still not helping and it still continues to shoot out of the grid. Also the distance value continues to constantly increase which looks quite weird to me. I have herewith attached the values and the cam chain files zed2_cam_imu_calib4-results-imucam.txt zed2_cam_imu_calib4-report-imucam.pdf

[yangyulin]

Have you fully excited the motion when doing calculations?It is clear that the IMU is not good, The scale correction for gyro and accel is too large

[goldbattle]

There are also some discussions about using ZED / ZED2 if you search the issues:

• https://github.com/rpng/open_vins/issues/52
• https://github.com/rpng/open_vins/issues/133
• https://github.com/rpng/open_vins/issues/157

[Aishkrish18]

Hey guys, thank you so much for your help and support regarding this issue. After a long period of digging deep into the parameters and calibration values, i was able to achieve a somewhat stable tracking. Although, it takes a long time to initialise even after i excite all the axes of IMU. Also, as mentioned in issue #367 the camera loses its tracking after tracking stable for a long time. In my case, the directions/axes were in opposite direction, so i think that still needs to be corrected. I do have another question were if its possible to create and save a map with this method or can we merely just get localisation? THanks a lot for your support!

[goldbattle]

This project is just about producing odometry, so saving and localizing against a map is out of the scope. You can checkout the ov_maplab project as an initial direction.

[goldbattle]

Also feel free to pose your ZED2 configs if you are able as it seems quite a few people have tried to use this device. Many thanks!