search

STAR-Center / VINS-RGBD

GNU General Public License v3.0
334 stars 82 forks source link

How to run this work using realsense d435i? #7

Open daijicheng opened 4 years ago

daijicheng commented 4 years ago

I tried to run this work using a realsense d435i, but I found that the raw imu output of realsense roswarpper is different to the imu data in the Handheld / Normal.bag you provided. The raw imu output is in "camera_imu_optical_frame", but it is in "camera_imu_frame" in the Handheld / Normal.bag. Could you tell me what is the difference between the two frames, or how can I convert the raw output imu data to the "camera_imu_frame" like the data you provided?

L1ttlewhite commented 4 years ago

Unfortunately, I faced this problem for a few days. It is the main issue why you or I can not apply this algorithm to our d435i well. It has a rotation between the /camera_imu_frame and the /camera_imu_optical_frame. This will cause a wrong point cloud mapping and a bad imu2cam frame transform if you don't fix the bug. I will try to fix it by changing the imu frame. Hope this can help you!

RigerLee commented 4 years ago

@daijicheng @L1ttlewhite We didn't use tf, so change of frameid makes no difference. One possible reason is that the d435i ros driver changed the transformation between image and imu. You can either checkout an old version of realsense driver or find the new transformation between image frame and imu frame(and then modify config file).

L1ttlewhite commented 4 years ago

@RigerLee I use realsensesdk v.2.31 ( the newest version). The transformation between image and imu is different, so we calibrate it by Kalibr. It works well now. Thanks for your great work :)

BarzelS commented 4 years ago

@RigerLee I use realsensesdk v.2.31 ( the newest version). The transformation between image and imu is different, so we calibrate it by Kalibr. It works well now. Thanks for your great work :)

Hi @L1ttlewhite, I'm also trying to run it with the D435i, can you please elaborate more about the process you did in order to fix this issue?

I'm getting large jumps in odomety and a lot of prints like: [ INFO] [1589471958.142266390]: IMU excitation not enouth! [ INFO] [1589471958.142819763]: Not enough features or parallax; Move device around

image

image

Have you encountered the same problems?

L1ttlewhite commented 4 years ago

@shirbarzel I can't see the images. The INFO messages seem that you need to move the device, the parallax is need for the VO. The important thing is to make the realsense_color_config.yaml suitable for your device. You can try
estimate_extrinsic: 2 for online calibration and follow the INFO

BarzelS commented 4 years ago

@shirbarzel I can't see the images. The INFO messages seem that you need to move the device, the parallax is need for the VO. The important thing is to make the realsense_color_config.yaml suitable for your device. You can try estimate_extrinsic: 2 for online calibration and follow the INFO

  1. Here above you said that there is a problem with the imu_link with the realsense sdk , how did you solve this issue? With the Kalibr tool? isn't this tool just give you more accurate parameters?
  2. What do you mean by making the realsense_color_config.yaml suitable, what is the process that I need to do in order to do it? Use the Kalibr?
arindamsaha81 commented 4 years ago

@RigerLee I use realsensesdk v.2.31 ( the newest version). The transformation between image and imu is different, so we calibrate it by Kalibr. It works well now. Thanks for your great work :)

Hi @L1ttlewhite Can you please describe the process of calibrating d435i using Kalibr. I have tried with Kalibr, but I guess my calibration is not correct.

Because when I calibrate first the camera, realsense produces RGB images which is not directly usable with Kalibr, so I had to change the RGB images to Monochrome images and create a new rosbag. Then the fx and fy is coming near to 545.2 but as we know the true fx and fy is about 611.18 and the reprojection error is more than 1 pixel. So I think my calibration is not very good. Afterwards when I calibrate the tranformation from imu to camera then also the reprojection error is more than 1 pixel. Another confusion is how get the values of accelerometer_noise_density, accelerometer_random_walk, gyroscope_noise_density, gyroscope_random_walk in the imu.yaml file. Did u use any other application to get those or anywhere are these given?

BarzelS commented 4 years ago

@RigerLee I use realsensesdk v.2.31 ( the newest version). The transformation between image and imu is different, so we calibrate it by Kalibr. It works well now. Thanks for your great work :)

Hi @L1ttlewhite Can you please describe the process of calibrating d435i using Kalibr. I have tried with Kalibr, but I guess my calibration is not correct.

Because when I calibrate first the camera, realsense produces RGB images which is not directly usable with Kalibr, so I had to change the RGB images to Monochrome images and create a new rosbag. Then the fx and fy is coming near to 545.2 but as we know the true fx and fy is about 611.18 and the reprojection error is more than 1 pixel. So I think my calibration is not very good. Afterwards when I calibrate the tranformation from imu to camera then also the reprojection error is more than 1 pixel. Another confusion is how get the values of accelerometer_noise_density, accelerometer_random_walk, gyroscope_noise_density, gyroscope_random_walk in the imu.yaml file. Did u use any other application to get those or anywhere are these given?

Hi @arindamsaha As explained here: https://github.com/iperper/Kimera-VIO-ROS/commit/ab90f0f7413458382903aa7ce1eb337deed3ddb4?short_path=853a910 You can use these scripts to get the imu intrinsics: (accelerometer_noise_density, accelerometer_random_walk, gyroscope_noise_density, gyroscope_random_walk)

They said you can also use this file to get this parameters but it will be more precise to use the scripts I'm mentioned above.

I did the same procedure but know I'm interested in how to get the parameters from Kalibr and update them in the relevant files in VINS? (I'm trying both VINS-FUSION and VINS-RGBD with the D435i)?

Is there any scripts that takes the output of kalibr(camchain-imu, results-imucam) and updates the relevant VINS config files:

Thanks

arindamsaha81 commented 4 years ago

Hi @DronistB Many thanks for your information. I am unaware about any script that copy the relevant data to proper place. I usually does that manually but I guess, you can always write your own script to do that.

Thanks

BarzelS commented 4 years ago

Hi @DronistB Many thanks for your information. I am unaware about any script that copy the relevant data to proper place. I usually does that manually but I guess, you can always write your own script to do that.

Thanks

Can you please shortly explain the process you are doing manually?