rpng / open_vins

An open source platform for visual-inertial navigation research.
https://docs.openvins.com
GNU General Public License v3.0
2.05k stars 615 forks source link

ROS1 Realsense D435i TF drift #441

Open VivekMangeUD opened 2 months ago

VivekMangeUD commented 2 months ago

Hello I am working on testing Realsense D435i with Openvins and I am facing drift issue with the TF. I have attached the screen recording, It shows the Rviz window. After the initialzation, With try_zupt = True. The TF is statioray and able to get the features, But when I move the camera, the TF has variable drift. How do I avoid the drift?

I have shared the config file. IMU bag file is 15hr long. I doubled down the noise and 10times the walk density.

kalibr_imu_chain

%YAML:1.0

imu0: T_i_b:

kalibr_imucam_chain

%YAML:1.0

cam0: T_cam_imu:

estimator_config

%YAML:1.0 # need to specify the file type at the top!

verbosity: "INFO" # ALL, DEBUG, INFO, WARNING, ERROR, SILENT

use_fej: true # if first-estimate Jacobians should be used (enable for good consistency) integration: "rk4" # discrete, rk4, analytical (if rk4 or analytical used then analytical covariance propagation is used) use_stereo: true # if we have more than 1 camera, if we should try to track stereo constraints between pairs max_cameras: 1 # how many cameras we have 1 = mono, 2 = stereo, >2 = binocular (all mono tracking)

calib_cam_extrinsics: true # if the transform between camera and IMU should be optimized R_ItoC, p_CinI calib_cam_intrinsics: true # if camera intrinsics should be optimized (focal, center, distortion) calib_cam_timeoffset: true # if timeoffset between camera and IMU should be optimized calib_imu_intrinsics: false # if imu intrinsics should be calibrated (rotation and skew-scale matrix) calib_imu_g_sensitivity: false # if gyroscope gravity sensitivity (Tg) should be calibrated

max_clones: 11 # how many clones in the sliding window max_slam: 50 # number of features in our state vector max_slam_in_update: 25 # update can be split into sequential updates of batches, how many in a batch max_msckf_in_update: 40 # how many MSCKF features to use in the update dt_slam_delay: 1 # delay before initializing (helps with stability from bad initialization...)

gravity_mag: 9.81 # magnitude of gravity in this location

feat_rep_msckf: "GLOBAL_3D" feat_rep_slam: "ANCHORED_MSCKF_INVERSE_DEPTH" feat_rep_aruco: "ANCHORED_MSCKF_INVERSE_DEPTH"

try_zupt: true zupt_chi2_multipler: 0 # set to 0 for only disp-based zupt_max_velocity: 0.1 zupt_noise_multiplier: 10 zupt_max_disparity: 0.5 # set to 0 for only imu-based zupt_only_at_beginning: false

init_window_time: 2.0 # how many seconds to collect initialization information init_imu_thresh: 1.5 # threshold for variance of the accelerometer to detect a "jerk" in motion init_max_disparity: 10.0 # max disparity to consider the platform stationary (dependent on resolution) init_max_features: 50 # how many features to track during initialization (saves on computation)

init_dyn_use: false # if dynamic initialization should be used init_dyn_mle_opt_calib: false # if we should optimize calibration during intialization (not recommended) init_dyn_mle_max_iter: 50 # how many iterations the MLE refinement should use (zero to skip the MLE) init_dyn_mle_max_time: 0.05 # how many seconds the MLE should be completed in init_dyn_mle_max_threads: 6 # how many threads the MLE should use init_dyn_num_pose: 6 # number of poses to use within our window time (evenly spaced) init_dyn_min_deg: 10.0 # orientation change needed to try to init

init_dyn_inflation_ori: 10 # what to inflate the recovered q_GtoI covariance by init_dyn_inflation_vel: 100 # what to inflate the recovered v_IinG covariance by init_dyn_inflation_bg: 10 # what to inflate the recovered bias_g covariance by init_dyn_inflation_ba: 100 # what to inflate the recovered bias_a covariance by init_dyn_min_rec_cond: 1e-12 # reciprocal condition number thresh for info inversion

init_dyn_bias_g: [ 0.0, 0.0, 0.0 ] # initial gyroscope bias guess init_dyn_bias_a: [ 0.0, 0.0, 0.0 ] # initial accelerometer bias guess

record_timing_information: false # if we want to record timing information of the method record_timing_filepath: "/tmp/traj_timing.txt" # https://docs.openvins.com/eval-timing.html#eval-ov-timing-flame

save_total_state: false filepath_est: "/tmp/ov_estimate.txt" filepath_std: "/tmp/ov_estimate_std.txt" filepath_gt: "/tmp/ov_groundtruth.txt"

use_klt: true # if true we will use KLT, otherwise use a ORB descriptor + robust matching num_pts: 200 # number of points (per camera) we will extract and try to track fast_threshold: 30 # threshold for fast extraction (warning: lower threshs can be expensive) grid_x: 5 # extraction sub-grid count for horizontal direction (uniform tracking) grid_y: 5 # extraction sub-grid count for vertical direction (uniform tracking) min_px_dist: 15 # distance between features (features near each other provide less information) knn_ratio: 0.70 # descriptor knn threshold for the top two descriptor matches track_frequency: 31.0 # frequency we will perform feature tracking at (in frames per second / hertz) downsample_cameras: false # will downsample image in half if true num_opencv_threads: 4 # -1: auto, 0-1: serial, >1: number of threads histogram_method: "HISTOGRAM" # NONE, HISTOGRAM, CLAHE

use_aruco: false num_aruco: 1024 downsize_aruco: true

up_msckf_sigma_px: 1 up_msckf_chi2_multipler: 1 up_slam_sigma_px: 1 up_slam_chi2_multipler: 1 up_aruco_sigma_px: 1 up_aruco_chi2_multipler: 1

use_mask: false

relative_config_imu: "kalibr_imu_chain.yaml" relative_config_imucam: "kalibr_imucam_chain.yaml"

Attached Video : https://drive.google.com/drive/folders/1ucrn0azjHuLtg9NBeXxr7varyWQujsrm?usp=sharing

Screenshot from 2024-05-01 19-54-11

VivekMangeUD commented 1 month ago

Hey guys

I have tried a lot of paramerters but I still have the drift. I have noticed a lot of issues raised on the repo about drift but most of them are silent. I found this issue https://github.com/rpng/open_vins/issues/427 which tells about the imu error, Mine is also very large. I have attached my yaml files and PDF in the below git hub with the bag files as well. Does anyone has any suggestion? https://drive.google.com/drive/folders/1ucrn0azjHuLtg9NBeXxr7varyWQujsrm?usp=sharing