quxichao commented 5 years ago

Hello, I recently used D435i to collect data for performance testing.

the rs_camera.launch is `

`

the realsense_stereo_imu_config.yaml is `%YAML:1.0

common parameters

support: 1 imu 1 cam; 1 imu 2 cam: 2 cam;

imu: 1
num_of_cam: 2

imu_topic: "/camera/imu" image0_topic: "/camera/infra1/image_rect_raw" image1_topic: "/camera/infra2/image_rect_raw" output_path: "/home/quxichao/catkin-ws/output/"

cam0_calib: "left.yaml" cam1_calib: "right.yaml" image_width: 640 image_height: 480

Extrinsic parameter between IMU and Camera.

estimate_extrinsic: 1 # 0 Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it.

1 Have an initial guess about extrinsic parameters. We will optimize around your initial guess.

body_T_cam0: !!opencv-matrix rows: 4 cols: 4 dt: d data: [ -5.7586305857286746e-03, -4.0463318787729019e-03, 9.9997523237933461e-01, 2.0329267950355900e-02, -9.9998287214160420e-01, -1.0224590553211677e-03, -5.7628118925283633e-03, 7.9325209639615653e-03, 1.0457519809151661e-03, -9.9999129084997906e-01, -4.0403746097850135e-03, 2.8559824645148020e-03, 0., 0., 0., 1. ]

body_T_cam1: !!opencv-matrix rows: 4 cols: 4 dt: d data: [ -1.0021770212322867e-03, 3.6313480322730518e-04, 9.9999943188700535e-01, 1.5285779565991807e-02, -9.9999216342926500e-01, -3.8303422615924010e-03, -1.0007788055728661e-03, -5.2435791444330505e-02, 3.8299766679101843e-03, -9.9999259827824449e-01, 3.6697063849344680e-04, 8.6931302450199057e-03, 0., 0., 0., 1. ]

Multiple thread support

multiple_thread: 1

feature traker paprameters

max_cnt: 150 # max feature number in feature tracking min_dist: 30 # min distance between two features freq: 10 # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image F_threshold: 1.0 # ransac threshold (pixel) show_track: 0 # publish tracking image as topic flow_back: 1 # perform forward and backward optical flow to improve feature tracking accuracy

optimization parameters

max_solver_time: 0.04 # max solver itration time (ms), to guarantee real time max_num_iterations: 8 # max solver itrations, to guarantee real time keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

imu parameters The more accurate parameters you provide, the better performance

acc_n: 0.1 # accelerometer measurement noise standard deviation. #0.2 0.04 gyr_n: 0.01 # gyroscope measurement noise standard deviation. #0.05 0.004 acc_w: 0.001 # accelerometer bias random work noise standard deviation. #0.002 gyr_w: 0.0001 # gyroscope bias random work noise standard deviation. #4.0e-5 g_norm: 9.805 # gravity magnitude

unsynchronization parameters

estimate_td: 1 # online estimate time offset between camera and imu td: 0.00 # initial value of time offset. unit: s. readed image clock + td = real image clock (IMU clock)

loop closure parameters

load_previous_pose_graph: 0 # load and reuse previous pose graph; load from 'pose_graph_save_path' pose_graph_save_path: "/home/quxichao/catkin-ws/output/pose_graph/" # save and load path save_image: 0 # save image in pose graph for visualization prupose; you can close this function by setting 0 `

When I run the code，the track_image is none,and the location is drifting

Wangfeng2394 commented 5 years ago

感觉遇到的问题跟你一样

pjrambo commented 5 years ago

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

Wangfeng2394 commented 5 years ago

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

pjrambo commented 5 years ago

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

Wangfeng2394 commented 5 years ago

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

But on my computer， VINS cant provide satisfactory result.
The path is very different from the real scene

Wangfeng2394 commented 5 years ago

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

I using dual-camera and Calibrate them by kalibr.

My cameras are 120degree angle.

Wangfeng2394 commented 5 years ago

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

I tried to calibrate my cameras by different types. such as mei & pinhole

Wangfeng2394 commented 5 years ago

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

Config file as follows, can you give me some tips and advice?: QQ图片20190623171838

shaoxiang commented 5 years ago

@quxichao @HKVINS89757 The same problem, has anyone solved it? I tried switching SDK to version 2.18.1, but it was worse. 我换了SDK版本后, 还是没有解决这个问题 Screenshot from 2019-07-16 14-12-54

Wangfeng2394 commented 5 years ago

@quxichao @HKVINS89757 The same problem, has anyone solved it? I tried switching SDK to version 2.18.1, but it was worse.

说中文

SainaRez commented 5 years ago

@pjrambo Which Intel firmware version are you guys using on D435i?

takmin commented 5 years ago

I have same problem with realsense D435i (Mono Camera + IMU). SDK version is 2.23.0

Here is a link to rosbag and configuration file: https://drive.google.com/open?id=1W9tkiHEgJiWnozo_7JZIxDuND5SF3lx1

MingheCao commented 5 years ago

Same problem with 2.25 SDK. Could anyone give a solution?

cichard commented 5 years ago

Hello 👋🏻 ! Many issues here .... First consider switching on the sdk 2.25 and ros corresponding 2.2.8 because they fix many problem on the imu part. Then use the last production firmware 11.2.6.250

-Then do not use their linear interpolation implementation : because as you will be able to see it is buggy ! It is a mixed interpolation between fast and slow stream so as you will be able to see time stamp are messed up !

Then consider using imu_tools to compute Allan parameters of your device : you will have higher value than those provided by the config file in the github repository of vins fusion : impressive -Then I will recommend you to use global time flag in the ros node of RealSense to get time of capture instead of time of reception message and better do not use ros node provided by realsense but one called rs4se which makes a good implementation for visual inertial system
use Kalibr to make the calibration between cameras and camera/imu
take into account that vins convention seems to wait for a z up body frame in the camera to body projection matrix
vins is waiting for imu data that is corrected for misalignment and bias, the realsense documentation do state that the operation is performed in the firmware after performing the es-imu-calibration script. But as I could have seen, there is a bug in the correction, so you will have to disable this correction by setting the option “enable motion correction “ and implement it by yourself in the realsense node ! Hopefully it can help anyone

wfg666 commented 4 years ago

I have just run vins successfully with a realsense D435i camera. Stereo + IMU config is fine. The success relies heavily on versions.

As a suggestion, you may try: ubuntu 18.04 + ROS melodic (I think 16.04+ROS Kinetic shoud also work) Linux kernel 4.15.0-72-generic (5.0 didn't work for me, be sure to reinstall librealsense2-dkms after changing kernel) librealsense2 2.19.0-0\~realsense0.615 librealsense2-dev 2.19.0-0\~realsense0.615 (2.19.2 didn't work for me) librealsense2-dkms 1.3.10-0ubuntu1 realsense2_camera 2.2.3 (just git checkout 2.2.3 and catkin_make) a realsense launch file at https://github.com/mhc994/VINS-Fusion/blob/wz_fly/config/realsense_d435i/vins_infra.launch (I disabled useless depth and color topics)

good luck

ceccocats commented 4 years ago

Hi @mhc994 Im trying d435i and succefully make VINS working with 1 imu and 1 camera but the stereo is not working. Im using SDK 2.36 and rs4se to grab the data. The calibration made with kalibr is good with very low errors, but still I have awful result when using stereo. I think that one issue can be the calibration of the IMU that is different from yours, my calib is the right one. As you can see the IMU position is rotated: Screenshot from 2020-08-19 15-45-44 Do you experienced the same issue with the different versions you used?

Im going to try your setup but I have problems to start my pc with linux kernel 4.15, can you tell me what type of problem you had with kernel 5?

thanks

cichard commented 4 years ago

Hi @mhc994 Im trying d435i and succefully make VINS working with 1 imu and 1 camera but the stereo is not working. Im using SDK 2.36 and rs4se to grab the data. The calibration made with kalibr is good with very low errors, but still I have awful result when using stereo. I think that one issue can be the calibration of the IMU that is different from yours, my calib is the right one. As you can see the IMU position is rotated: Do you experienced the same issue with the different versions you used?

Im going to try your setup but I have problems to start my pc with linux kernel 4.15, can you tell me what type of problem you had with kernel 5?

thanks

As @mhc994 confirms it, and as i said in my previous post VINS convention waits for IMU with Z up so apply the corresponding rotation to your Kalibr matrix parameter.

dyb678 commented 4 weeks ago

@HKVINS89757 同样的问题，有人解决过吗？我尝试将 SDK 切换到版本 2.18.1，但情况更糟。我换了SDK版本后，还是没有解决这个问题

Hello~do you solve this problem?Actually , I find that the mininum version for D435i is 2.36.0. How can I install 2.18.0?

HKUST-Aerial-Robotics / VINS-Fusion

A large drifting with Realsense D435i #53

common parameters

support: 1 imu 1 cam; 1 imu 2 cam: 2 cam;

Extrinsic parameter between IMU and Camera.

1 Have an initial guess about extrinsic parameters. We will optimize around your initial guess.

Multiple thread support

feature traker paprameters

optimization parameters

imu parameters The more accurate parameters you provide, the better performance

unsynchronization parameters

loop closure parameters