realsense d435i with tx2

[nuyhead-hwang]

Hello.

First of all, Thank you for your great working .

We experimented Open VINS on a Jetson TX2 using the Intel Realsense D-435i.

We are testing Open VINS for the Drone's Position Flight.

By the way, when we tested Open VINS last year (in commit f56cac358509226074f5fbe0d956001464006d34), the pose tracking is working well.

But when we tested Open VINS in recent commit (in commit e516f7c564c2159b7edfb42a8230b732bf6aebc2), pose tracking is more drift aware (especially when device is frozen).

like below image In Current Depth image, reprojected points are diverged.

Below is our launch file URL of open vins [https://gist.github.com/vislero/9aeb297d2cc63423fe7089a16309aff7]

What is the main difference between two version?

And when we turned on the 'zupt' mode in launch script. the 'zupt' module is working not perfectly. How to adjust the parameters to work fine ?

Thanks.

Regards.

[goldbattle]

It is tough to pick a threshold for the zero velocity update. If you are using SLAM features (it looks like you are), you can likely get away with not using the zero velocity update at all. The update will work better with a higher quality IMU, so I am not sure if the d435i will give reasonable results or not (have never used myself).

I assume the issue is that when you place the unit down it is flying away? Are you getting any SLAM feature updates during this time?

[nuyhead-hwang]

@goldbattle Thank you for kind reply.

Yes, I thought so.

When i stop moving d435.

The reprojected points are flying away like in the gif picture below.

And, it seems that the SLAM feature has been updated during this period.

What is the problem?

As mentioned earlier, it works perfectly in Oepn VINS v.2.2 using the same launch file (parameters). launch file : (https://gist.github.com/vislero/9aeb297d2cc63423fe7089a16309aff7)

Thank you.

[goldbattle]

Can you possibly upload a bag for me to debug with in the upcoming week? This shouldn't be the case, so I am not sure if there was a regression.

[nuyhead-hwang]

Hello @goldbattle

we uploaded the bag file in below url.

rosbag_file

Thank you !

[goldbattle]

It seems to work out of the box on the bag you sent. I just pushed some new zero velocity detection code which uses the image disparity now so you might want to test that. https://gist.github.com/goldbattle/38633c3ccc813787b480f65c80df1661

[nuyhead-hwang]

That's a great performance !

Now, reprojected points not diverge in current depth view.

Thank you very much.

By the way, Sometimes the pose estimated in the initialization phase diverge. Do you have some tips for robust initialization?

And, in a stationary situation, when I turned on the zupt module and started openvins, the poses diverge. How to fine tune the parameters about the zero velocity detection?

[nuyhead-hwang]

Sorry. We had not updated Open VINS.

Updating the latest version of Open VINS solved all problems

Now let's close this issue.

Thank you very much so far.

[Majed-Alsubaie]

Hi, @vislero I would like to ask how did you calibrate the d435i ? did you use official Realsensce calibration tool ?

Hi @goldbattle , I am planning to integrate Openvins with real drone , Do you have any advice , guided , certain test to check before first flight ?

• Did you try to use OpenVins with Rtabmap ? any issues with it ?

[nuyhead-hwang]

I used Kalibr to calibrate intrinsic, extrinsic parameters of d435i.

[ClementLASSENA]

Hi @vislero, Does your configuration (d435 + open_vins) work in mono? Indeed, I am trying to set a similar configuration but I am not sure that the mono-MSCKF can converge with this type of consumer grade IMU.

[mzahana]

Hi @vislero

Where you able to use open_vins in flight ?

I am trying to set it up on a drone usgin Jetson Orin NX ad Realsense D455 camera.

I face issues during the initialization of the filter. What parameters did you use in the estimator_config.yaml?

[lesliennz]

I used Kalibr to calibrate intrinsic, extrinsic parameters of d435i.

[lesliennz]

That's a great performance !

Now, reprojected points not diverge in current depth view.

Thank you very much.

By the way, Sometimes the pose estimated in the initialization phase diverge. Do you have some tips for robust initialization?

And, in a stationary situation, when I turned on the zupt module and started openvins, the poses diverge. How to fine tune the parameters about the zero velocity detection?

Hello, may I ask if you can provide the value of your initialization parameters? I am also using d435i, but it can not be well refined initialization, which is the following parameters: init_window_time: init_imu_thresh init_max_disparity init_max_features Thank you so much.