lukasvst / dm-vio

Source code for the paper DM-VIO: Delayed Marginalization Visual-Inertial Odometry
GNU General Public License v3.0
1.07k stars 187 forks source link

run dm-vio with own data #36

Closed LvDingzhou closed 1 year ago

LvDingzhou commented 1 year ago

To run dm-vio with own data, I made camera.txt, imu.txt, camchain.yaml,and times.txt files. (I've runned interpolate_imu_file) And I runned dm-vio with the files. But dm-vio doesn't work. Any suggestion for this problem?

Additionally, running with the --noimu option works well.

bin/dmvio_dataset files=/home/intel/bagfiles/dmvio_data/image1 imuFile=/home/intel/bagfiles/dmvio_data/image1/imu.txt calib=/home/intel/bagfiles/dmvio_data/configs/camera.txt imuCalib=/home/intel/bagfiles/dmvio_data/configs/camchain.yaml mode=1 preset=0 nogui=0 resultsPrefix=/home/intel/bagfiles/dmvio_data/result/ settingsFile=/home/intel/bagfiles/dmvio_data/configs/tumvi.yaml

PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== PRESET Settings: =============== DEFAULT settings:

  • no real-time enforcing
  • 2000 active points
  • 5-7 active frames
  • 1-6 LM iteration each KF
  • original image resolution

    Loading settings from yaml file: /home/intel/bagfiles/dmvio_data/configs/tumvi.yaml! Loading IMU parameter file at: /home/intel/bagfiles/dmvio_data/configs/camchain.yaml Used T_cam_imu: 0.999416 0.00818122 -0.0331675 -0.0273869 -0.00817461 0.999967 0.000334768 -0.00462149 0.0331691 -6.34414e-05 0.99945 -0.0160095 0 0 0 1 Used noise values: 0.043 0.0011 0.224 0.0128 Settings: accelerometer_noise_density: 0.224 accelerometer_random_walk: 0.043 addVisualToCoarseGraphIfTrackingBad: 0 alwaysCanBreakIMU: 0 baToCoarseAccBiasVariance: 1000 baToCoarseGyrBiasVariance: 0.05 baToCoarsePoseVariance: 0.1 baToCoarseRotVariance: 1 baToCoarseVelVariance: 0.1 calib: /home/intel/bagfiles/dmvio_data/configs/camera.txt dynamicWeightRMSEThresh: 8 end: 100000 files: /home/intel/bagfiles/dmvio_data/image1 fixKeyframeDuringCoarseTracking: 1 gamma: generalScaleIntervalSize: 60 gravityDirectionFixZ: 1 gtFile: gyroscope_noise_density: 0.0128 gyroscope_random_walk: 0.0011 imuCalib: /home/intel/bagfiles/dmvio_data/configs/camchain.yaml imuFile: /home/intel/bagfiles/dmvio_data/image1/imu.txt init_coarseScaleUncertaintyThresh: 1 init_conversionType: 0 init_disableVIOUntilFirstInit: 1 init_fixPoses: 1 init_initDSOParams: 1 init_lambdaLowerBound: 1e-16 init_maxNumPoses: 100 init_multipleBiases: 0 init_multithreadedInitDespiteNonRT: 0 init_onlyKFs: 1 init_pgba_conversionType: 0 init_pgba_delay: 100 init_pgba_prepareGraphAddDelValues: 0 init_pgba_prepareGraphAddFactors: 0 init_pgba_priorExtrinsicsRot: 0.01 init_pgba_priorExtrinsicsTrans: 0.1 init_pgba_priorGravityDirection: 0.4 init_pgba_priorGravityDirectionZ: 0.0001 init_pgba_reinitScaleUncertaintyThresh: 0.5 init_pgba_scaleUncertaintyThresh: 1 init_pgba_skipFirstKFs: 0 init_priorExtrinsicsRot: 0.01 init_priorExtrinsicsTrans: 0.1 init_priorGravityDirection: 0.4 init_priorGravityDirectionZ: 0.0001 init_priorRotSigma: 1e-05 init_priorTransSigma: 1e-05 init_requestFullResetErrorThreshold: -1 init_requestFullResetNormalizedErrorThreshold: -1 init_scalePriorAfterInit: 0 init_secondthreshGravdir: 1000 init_secondthreshScale: 1e+07 init_threshGravdir: 1000 init_threshScale: 1.02 init_transitionModel: 2 init_updatePoses: 1 integration_sigma: 0.316227 maxFrameEnergyThreshold: 5000 maxPreloadImages: 0 maxTimeBetweenInitFrames: 100000 numMeasurementsGravityInit: 40 preload: 0 priorExtrinsicsRot: 0.01 priorExtrinsicsTrans: 0.1 priorGravityDirection: 0.4 priorGravityDirectionZ: 0.0001 resultsPrefix: /home/intel/bagfiles/dmvio_data/result/ reverse: 0 sampleoutput: 0 setting_forceNoKFTranslationThresh: 0 setting_maxOptIterations: 6 setting_minFramesBetweenKeyframes: -0.5 setting_minIdepth: 0.02 setting_minOptIterations: 1 setting_optGravity: 1 setting_optIMUExtrinsics: 0 setting_optScaleBA: 1 setting_prior_bias: 0 setting_prior_velocity: 0 setting_scaleFixTH: 0 setting_solverMode: 2048 setting_transferCovToCoarse: 1 setting_visualOnlyAfterScaleFixing: 0 setting_weightDSOCoarse: 0.001 setting_weightDSOToGTSAM: 1.66667e-05 setting_weightZeroPriorDSOInitX: 0 setting_weightZeroPriorDSOInitY: 0 skipFirstKeyframe: 0 speed: 0 start: 0 transferCovToCoarseMultiplier: 1 updateDynamicWeightDuringOptimization: 1 use16Bit: 0 useScaleDiagonalHack: 0 vignette: Reading Calibration from file /home/intel/bagfiles/dmvio_data/configs/camera.txt ... found! Input resolution: 1280 720 In: 644.000000 653.000000 665.000000 378.000000 -0.039000 Out: Rectify Crop Output resolution: 1280 720 finding CROP optimal new model! initial range: x: -1.0429 - 0.9629; y: -0.5846 - 0.5274! iteration 00001: range: x: -1.0377 - 0.9581; y: -0.5846 - 0.5274! iteration 00002: range: x: -1.0325 - 0.9533; y: -0.5846 - 0.5274! iteration 00003: range: x: -1.0325 - 0.9533; y: -0.5817 - 0.5248! iteration 00004: range: x: -1.0325 - 0.9533; y: -0.5788 - 0.5222! iteration 00005: range: x: -1.0325 - 0.9533; y: -0.5788 - 0.5222!

Rectified Kamera Matrix: 644.056 0 665.003 0 653.091 377.981 0 0 1

NO PHOTOMETRIC Calibration! Reading Photometric Calibration from file PhotometricUndistorter: Could not open file! set timestamps and exposures to zero! set EXPOSURES to zero! got 3849 images and 0 timestamps and 0 exposures.! ImageFolderReader: got 3849 files in /home/intel/bagfiles/dmvio_data/image1! IMU Id: 1672040735245689631 段错误 (核心已转储)

LvDingzhou commented 1 year ago

ok the problem as follows ↓

got 3849 images and 0 timestamps and 0 exposures.!

the "times.txt" should be placed with the same path as "images" e.x.

$ls $images times.txt

LvDingzhou commented 1 year ago

then,I meet a new problem

bin/dmvio_dataset \

files=/home/intel/bagfiles/dmvio_data/image1 \ imuFile=/home/intel/bagfiles/dmvio_data/imu.txt \ calib=/home/intel/bagfiles/dmvio_data/configs/camera.txt \ imuCalib=/home/intel/bagfiles/dmvio_data/configs/camchain.yaml \ mode=1 \ preset=0 \ nogui=0 \ resultsPrefix=/home/intel/bagfiles/dmvio_data/result/ \ settingsFile=/home/intel/bagfiles/dmvio_data/configs/tumvi.yaml \

PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== PRESET Settings: =============== DEFAULT settings:

  • no real-time enforcing
  • 2000 active points
  • 5-7 active frames
  • 1-6 LM iteration each KF
  • original image resolution

    Loading settings from yaml file: /home/intel/bagfiles/dmvio_data/configs/tumvi.yaml! Loading IMU parameter file at: /home/intel/bagfiles/dmvio_data/configs/camchain.yaml Used T_cam_imu: 0.999416 0.00818122 -0.0331675 -0.0273869 -0.00817461 0.999967 0.000334768 -0.00462149 0.0331691 -6.34414e-05 0.99945 -0.0160095 0 0 0 1 Used noise values: 0.043 0.0011 0.224 0.0128 Settings: accelerometer_noise_density: 0.224 accelerometer_random_walk: 0.043 addVisualToCoarseGraphIfTrackingBad: 0 alwaysCanBreakIMU: 0 baToCoarseAccBiasVariance: 1000 baToCoarseGyrBiasVariance: 0.05 baToCoarsePoseVariance: 0.1 baToCoarseRotVariance: 1 baToCoarseVelVariance: 0.1 calib: /home/intel/bagfiles/dmvio_data/configs/camera.txt dynamicWeightRMSEThresh: 8 end: 100000 files: /home/intel/bagfiles/dmvio_data/image1 fixKeyframeDuringCoarseTracking: 1 gamma: generalScaleIntervalSize: 60 gravityDirectionFixZ: 1 gtFile: gyroscope_noise_density: 0.0128 gyroscope_random_walk: 0.0011 imuCalib: /home/intel/bagfiles/dmvio_data/configs/camchain.yaml imuFile: /home/intel/bagfiles/dmvio_data/imu.txt init_coarseScaleUncertaintyThresh: 1 init_conversionType: 0 init_disableVIOUntilFirstInit: 1 init_fixPoses: 1 init_initDSOParams: 1 init_lambdaLowerBound: 1e-16 init_maxNumPoses: 100 init_multipleBiases: 0 init_multithreadedInitDespiteNonRT: 0 init_onlyKFs: 1 init_pgba_conversionType: 0 init_pgba_delay: 100 init_pgba_prepareGraphAddDelValues: 0 init_pgba_prepareGraphAddFactors: 0 init_pgba_priorExtrinsicsRot: 0.01 init_pgba_priorExtrinsicsTrans: 0.1 init_pgba_priorGravityDirection: 0.4 init_pgba_priorGravityDirectionZ: 0.0001 init_pgba_reinitScaleUncertaintyThresh: 0.5 init_pgba_scaleUncertaintyThresh: 1 init_pgba_skipFirstKFs: 0 init_priorExtrinsicsRot: 0.01 init_priorExtrinsicsTrans: 0.1 init_priorGravityDirection: 0.4 init_priorGravityDirectionZ: 0.0001 init_priorRotSigma: 1e-05 init_priorTransSigma: 1e-05 init_requestFullResetErrorThreshold: -1 init_requestFullResetNormalizedErrorThreshold: -1 init_scalePriorAfterInit: 0 init_secondthreshGravdir: 1000 init_secondthreshScale: 1e+07 init_threshGravdir: 1000 init_threshScale: 1.02 init_transitionModel: 2 init_updatePoses: 1 integration_sigma: 0.316227 maxFrameEnergyThreshold: 5000 maxPreloadImages: 0 maxTimeBetweenInitFrames: 100000 numMeasurementsGravityInit: 40 preload: 0 priorExtrinsicsRot: 0.01 priorExtrinsicsTrans: 0.1 priorGravityDirection: 0.4 priorGravityDirectionZ: 0.0001 resultsPrefix: /home/intel/bagfiles/dmvio_data/result/ reverse: 0 sampleoutput: 0 setting_forceNoKFTranslationThresh: 0 setting_maxOptIterations: 6 setting_minFramesBetweenKeyframes: -0.5 setting_minIdepth: 0.02 setting_minOptIterations: 1 setting_optGravity: 1 setting_optIMUExtrinsics: 0 setting_optScaleBA: 1 setting_prior_bias: 0 setting_prior_velocity: 0 setting_scaleFixTH: 0 setting_solverMode: 2048 setting_transferCovToCoarse: 1 setting_visualOnlyAfterScaleFixing: 0 setting_weightDSOCoarse: 0.001 setting_weightDSOToGTSAM: 1.66667e-05 setting_weightZeroPriorDSOInitX: 0 setting_weightZeroPriorDSOInitY: 0 skipFirstKeyframe: 0 speed: 0 start: 0 transferCovToCoarseMultiplier: 1 updateDynamicWeightDuringOptimization: 1 use16Bit: 0 useScaleDiagonalHack: 0 vignette: Reading Calibration from file /home/intel/bagfiles/dmvio_data/configs/camera.txt ... found! Input resolution: 1280 720 In: 644.617394 652.752771 665.397770 378.391226 -0.039464 Out: Rectify Crop Output resolution: 1280 720 finding CROP optimal new model! initial range: x: -1.0425 - 0.9613; y: -0.5854 - 0.5270! iteration 00001: range: x: -1.0373 - 0.9565; y: -0.5854 - 0.5270! iteration 00002: range: x: -1.0321 - 0.9517; y: -0.5854 - 0.5270! iteration 00003: range: x: -1.0321 - 0.9517; y: -0.5825 - 0.5244! iteration 00004: range: x: -1.0321 - 0.9517; y: -0.5796 - 0.5218! iteration 00005: range: x: -1.0321 - 0.9517; y: -0.5796 - 0.5218!

Rectified Kamera Matrix: 644.705 0 665.415 0 652.854 378.366 0 0 1

NO PHOTOMETRIC Calibration! Reading Photometric Calibration from file PhotometricUndistorter: Could not open file! set EXPOSURES to zero! got 3847 images and 3847 timestamps and 0 exposures.! ImageFolderReader: got 3847 files in /home/intel/bagfiles/dmvio_data/image1! IMU Id: 1672040735245689631 IMU-data too old -> skipping frame using pyramid levels 0 to 4. coarsest resolution: 80 x 45! START PANGOLIN! Using setting_minFramesBetweenKeyframes=0.5 because of non-realtime mode. Switching to initializer state: CoarseIMUInit PixelSelector: Using block sizes: 16, 16 Loading gt data terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc 已放弃 (核心已转储)

LvDingzhou commented 1 year ago

I solved this issue by adding

start 2

in the command line. e.x.

bin/dmvio_dataset \ files=/home/intel/bagfiles/dmvio_data/image1 imuFile=/home/intel/bagfiles/dmvio_data/imu.txt calib=/home/intel/bagfiles/dmvio_data/configs/camera.txt imuCalib=/home/intel/bagfiles/dmvio_data/configs/camchain.yaml mode=1 preset=0 nogui=0 resultsPrefix=/home/intel/bagfiles/dmvio_data/result/ settingsFile=/home/intel/bagfiles/dmvio_data/configs/tumvi.yaml start 2