search

lukasvst / dm-vio

Source code for the paper DM-VIO: Delayed Marginalization Visual-Inertial Odometry
GNU General Public License v3.0
1.05k stars 184 forks source link

Porting to Windows, crash on init #43

Open antithing opened 1 year ago

antithing commented 1 year ago

Hi, I am porting this to Windows, and I have everything compiling and launching (live with a T265) however, I see the following error, and the system crashes when it tries to run. dmvio.exe useimu=1 mode=0 preset=1 nogui=0 quiet=1 start=2 settingsFile=configs/t265_noise_tumvi.yaml resultsPrefix=/results/ calibSavePath=/calib.txt vignette=configs/realsense/vignette_t265.png gamma=configs/pcalib_linear_8bit.txt

gives me:


Enabling IMU integration!
PHOTOMETRIC MODE WITH CALIBRATION!

=============== PRESET Settings: ===============
DEFAULT settings:
- 1x real-time enforcing
- 2000 active points
- 5-7 active frames
- 1-6 LM iteration each KF
- original image resolution
==============================================
QUIET MODE, I'll shut up!
Loading settings from yaml file: configs/t265_noise_tumvi.yaml!
Settings:
accelerometer_noise_density: 0.224
accelerometer_random_walk: 0.043
addVisualToCoarseGraphIfTrackingBad: 0
alwaysCanBreakIMU: 0
baToCoarseAccBiasVariance: 1000
baToCoarseGyrBiasVariance: 0.05
baToCoarsePoseVariance: 0.1
baToCoarseRotVariance: 1
baToCoarseVelVariance: 0.1
calib:
calibSavePath: /FACTORY_CALIBRATION_WILL_BE_SAVED_HERE.txt
camchainSavePath:
dynamicWeightRMSEThresh: 8
fixKeyframeDuringCoarseTracking: 1
gamma: configs/pcalib_linear_8bit.txt
generalScaleIntervalSize: 60
gravityDirectionFixZ: 1
gyroscope_noise_density: 0.0128
gyroscope_random_walk: 0.0011
imuCalib:
init_coarseScaleUncertaintyThresh: 5
init_conversionType: 0
init_disableVIOUntilFirstInit: 1
init_fixPoses: 1
init_initDSOParams: 1
init_lambdaLowerBound: 1e-16
init_maxNumPoses: 100
init_multipleBiases: 0
init_multithreadedInitDespiteNonRT: 0
init_onlyKFs: 1
init_pgba_conversionType: 0
init_pgba_delay: 100
init_pgba_prepareGraphAddDelValues: 0
init_pgba_prepareGraphAddFactors: 0
init_pgba_priorExtrinsicsRot: 0.01
init_pgba_priorExtrinsicsTrans: 0.1
init_pgba_priorGravityDirection: 0.4
init_pgba_priorGravityDirectionZ: 0.0001
init_pgba_reinitScaleUncertaintyThresh: 1
init_pgba_scaleUncertaintyThresh: 5
init_pgba_skipFirstKFs: 1
init_priorExtrinsicsRot: 0.01
init_priorExtrinsicsTrans: 0.1
init_priorGravityDirection: 0.4
init_priorGravityDirectionZ: 0.0001
init_priorRotSigma: 1e-05
init_priorTransSigma: 1e-05
init_requestFullResetErrorThreshold: -1
init_requestFullResetNormalizedErrorThreshold: 0.1
init_scalePriorAfterInit: 0
init_secondthreshGravdir: 1000
init_secondthreshScale: 1e+07
init_threshGravdir: 1000
init_threshScale: 1.02
init_transitionModel: 2
init_updatePoses: 1
integration_sigma: 0.316227
maxFrameEnergyThreshold: 5000
maxSkipFramesFullReset: -1
maxSkipFramesVisualInertial: 2
maxSkipFramesVisualInit: 0
maxSkipFramesVisualOnlyMode: 1
maxTimeBetweenInitFrames: 1
minQueueSizeForSkipping: 2
normalizeCamSize: 0.2
numMeasurementsGravityInit: 40
preload: 1
priorExtrinsicsRot: 0.01
priorExtrinsicsTrans: 0.1
priorGravityDirection: 0.4
priorGravityDirectionZ: 0.0001
resultsPrefix: /PATH_TO_RESULTS/
saveDatasetPath:
setting_forceNoKFTranslationThresh: 0
setting_maxOptIterations: 6
setting_minFramesBetweenKeyframes: -0.5
setting_minIdepth: 0.02
setting_minOptIterations: 1
setting_optGravity: 1
setting_optIMUExtrinsics: 0
setting_optScaleBA: 1
setting_prior_bias: 0
setting_prior_velocity: 0
setting_scaleFixTH: 0
setting_solverMode: 2048
setting_transferCovToCoarse: 1
setting_visualOnlyAfterScaleFixing: 0
setting_weightDSOCoarse: 0.001
setting_weightDSOToGTSAM: 1.66667e-05
setting_weightZeroPriorDSOInitX: 0
setting_weightZeroPriorDSOInitY: 0
skipFirstKeyframe: 0
skipFramesVisualOnlyDelay: 30
speed: 1
start: 2
transferCovToCoarseMultiplier: 1
updateDynamicWeightDuringOptimization: 1
useScaleDiagonalHack: 0
vignette: configs/realsense/vignette_t265.png
Saving camera calibration to /FACTORY_CALIBRATION_WILL_BE_SAVED_HERE.txt
Sensor Tracking Module. Supported options:
        Exposure
        Gain
        Enable Auto Exposure
        Frames Queue Size
        Asic Temperature
        Motion Module Temperature
        Enable Mapping
        Enable Relocalization
        Enable Pose Jumping
        Enable Dynamic Calibration
        Enable Map Preservation
Device Intel RealSense T265 connected
Gyro Matrix
   0.994002           0           0  0.00232676
          0    0.994022           0 0.000956401
          0           0     0.99584 0.000545991
Gyro noise var: 0.00226893
0.00226893
0.00226893 bias var: 0.000707107
0.000707107
0.000707107
Accel noise var: 0.00818245
0.00818245
0.00818245 bias var: 0.01
0.01
0.01
T_imu_cam:    -0.999978 -0.000865133   0.00664827    0.0106998
 0.000853561    -0.999998  -0.00174326 -9.13311e-06
  0.00664977  -0.00173754     0.999976 -7.11525e-05
           0            0            0            1
T_cam_imu:    -0.999978  0.000853561   0.00664977       0.0107
-0.000865133    -0.999998  -0.00173754 -8.60805e-13
  0.00664827  -0.00174326     0.999976  2.30347e-13
           0            0            0            1
Reading Calibration from file /FACTORY_CALIBRATION_WILL_BE_SAVED_HERE.txt ... found!
Creating KannalaBrandt undistorter
Input resolution: 848 800
In: KannalaBrandt 283.915000 285.098000 424.730000 401.469000 -0.001734 0.037912 -0.035763 0.005849
Out: 0.200000 0.200000 0.499000 0.499000 0.000000
Output resolution: 848 800

Rectified Kamera Matrix:
  169.6       0 422.652
      0     160   398.7
      0       0       1

Reading Photometric Calibration from file configs/pcalib_linear_8bit.txt
Reading Vignette Image from configs/realsense/vignette_t265.png
Successfully read photometric calibration!
using pyramid levels 0 to 4. coarsest resolution: 53 x 50!
Using factory IMU calibration!
START PANGOLIN!
Switching to initializer state: RealtimeCoarseIMUInitState
PixelSelector: Using block sizes: 16, 16
PixelSelector: Using block sizes: 16, 16
InitTimeBetweenFrames: 0.0333333
InitTimeBetweenFrames: 0.0666666
InitTimeBetweenFrames: 0.0999999
InitTimeBetweenFrames: 0.133333
InitTimeBetweenFrames: 0.166667
InitTimeBetweenFrames: 0.2
InitTimeBetweenFrames: 0.233334
InitTimeBetweenFrames: 0.266667
Scaling with rescaleFactor: 5.14
INITIALIZE FROM INITIALIZER (966 pts)!
WARNING: Not sending frame, because it does not have IMU data yet.
WARNING: Not sending frame, because it does not have IMU data yet.
WARNING: Not sending frame, because it does not have IMU data yet.

And then the application crashes. What can i look at to solve this?

Thanks!

lukasvst commented 1 year ago

Does the version on datasets (e.g. on TUM-VI corridor 1) work properly or does it also crash?

It looks like it tries to save the calibration and results in the root directory of your drive, I'm not sure if it has write access on Windows for this directory.

Otherwise the best way to debug this error would be to run it with the debugger and check the stacktrace when the error happens.

antithing commented 1 year ago

Hi @lukasvst thank you for getting back to me. Running in Debug gives me:

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file
PhotometricUndistorter: Could not open file!
using pyramid levels 0 to 4. coarsest resolution: 53 x 50!
Using factory IMU calibration!
START PANGOLIN!
Switching to initializer state: RealtimeCoarseIMUInitState
'rm' is not recognized as an internal or external command,
operable program or batch file.
A subdirectory or file logs already exists.
'rm' is not recognized as an internal or external command,
operable program or batch file.
A subdirectory or file mats already exists.
PixelSelector: Using block sizes: 16, 16
SKIPPING 4 FRAMES! frames remaining in queue: 0
PixelSelector: Using block sizes: 16, 16
SKIPPING 0 FRAMES! frames remaining in queue: 4
In: D:\thirdParty\Pangolin-master\include\pangolin/gl/gl.hpp, line 214
InitTimeBetweenFrames: 0.0333335
SKIPPING 0 FRAMES! frames remaining in queue: 5
InitTimeBetweenFrames: 0.0666668
SKIPPING 0 FRAMES! frames remaining in queue: 6
InitTimeBetweenFrames: 0.1
SKIPPING 0 FRAMES! frames remaining in queue: 8
InitTimeBetweenFrames: 0.133334
SKIPPING 0 FRAMES! frames remaining in queue: 9
InitTimeBetweenFrames: 0.166667
SKIPPING 0 FRAMES! frames remaining in queue: 11
InitTimeBetweenFrames: 0.200001
SKIPPING 0 FRAMES! frames remaining in queue: 13
InitTimeBetweenFrames: 0.233334
SKIPPING 0 FRAMES! frames remaining in queue: 15
InitTimeBetweenFrames: 0.266667
SKIPPING 0 FRAMES! frames remaining in queue: 16
InitTimeBetweenFrames: 0.300001
SKIPPING 0 FRAMES! frames remaining in queue: 17
InitTimeBetweenFrames: 0.333334
SKIPPING 0 FRAMES! frames remaining in queue: 21
InitTimeBetweenFrames: 0.366668
SKIPPING 0 FRAMES! frames remaining in queue: 23
InitTimeBetweenFrames: 0.400001
SKIPPING 0 FRAMES! frames remaining in queue: 24
InitTimeBetweenFrames: 0.433334
SKIPPING 0 FRAMES! frames remaining in queue: 26
InitTimeBetweenFrames: 0.466668
SKIPPING 0 FRAMES! frames remaining in queue: 28
InitTimeBetweenFrames: 0.500001
SKIPPING 0 FRAMES! frames remaining in queue: 30
InitTimeBetweenFrames: 0.533334
SKIPPING 0 FRAMES! frames remaining in queue: 31
InitTimeBetweenFrames: 0.566668
SKIPPING 0 FRAMES! frames remaining in queue: 32
InitTimeBetweenFrames: 0.600001
SKIPPING 0 FRAMES! frames remaining in queue: 35
InitTimeBetweenFrames: 0.633334
SKIPPING 0 FRAMES! frames remaining in queue: 38
InitTimeBetweenFrames: 0.666668
SKIPPING 0 FRAMES! frames remaining in queue: 38
InitTimeBetweenFrames: 0.700001
SKIPPING 0 FRAMES! frames remaining in queue: 39
InitTimeBetweenFrames: 0.733335
SKIPPING 0 FRAMES! frames remaining in queue: 42
InitTimeBetweenFrames: 0.766668
SKIPPING 0 FRAMES! frames remaining in queue: 45
InitTimeBetweenFrames: 0.800001
SKIPPING 0 FRAMES! frames remaining in queue: 46
InitTimeBetweenFrames: 0.833335
SKIPPING 0 FRAMES! frames remaining in queue: 48
InitTimeBetweenFrames: 0.866668
SKIPPING 0 FRAMES! frames remaining in queue: 51
InitTimeBetweenFrames: 0.900002
SKIPPING 0 FRAMES! frames remaining in queue: 53
InitTimeBetweenFrames: 0.933161
SKIPPING 0 FRAMES! frames remaining in queue: 55
InitTimeBetweenFrames: 0.966494
SKIPPING 0 FRAMES! frames remaining in queue: 57
InitTimeBetweenFrames: 0.999828
SKIPPING 0 FRAMES! frames remaining in queue: 59
InitTimeBetweenFrames: 1.03316
SKIPPING 0 FRAMES! frames remaining in queue: 61
InitTimeBetweenFrames: 1.06649
SKIPPING 0 FRAMES! frames remaining in queue: 62
InitTimeBetweenFrames: 1.09983
SKIPPING 0 FRAMES! frames remaining in queue: 63
InitTimeBetweenFrames: 1.13316
SKIPPING 0 FRAMES! frames remaining in queue: 66
InitTimeBetweenFrames: 1.16649
SKIPPING 0 FRAMES! frames remaining in queue: 69
InitTimeBetweenFrames: 1.19983
SKIPPING 0 FRAMES! frames remaining in queue: 70
InitTimeBetweenFrames: 1.23316
SKIPPING 0 FRAMES! frames remaining in queue: 71
InitTimeBetweenFrames: 1.26649
SKIPPING 0 FRAMES! frames remaining in queue: 74
InitTimeBetweenFrames: 1.29983
SKIPPING 0 FRAMES! frames remaining in queue: 76
InitTimeBetweenFrames: 1.33316
SKIPPING 0 FRAMES! frames remaining in queue: 77
InitTimeBetweenFrames: 1.3665
SKIPPING 0 FRAMES! frames remaining in queue: 79
InitTimeBetweenFrames: 1.39983
SKIPPING 0 FRAMES! frames remaining in queue: 81
InitTimeBetweenFrames: 1.43316
SKIPPING 0 FRAMES! frames remaining in queue: 84
InitTimeBetweenFrames: 1.4665
SKIPPING 0 FRAMES! frames remaining in queue: 85
InitTimeBetweenFrames: 1.49983
SKIPPING 0 FRAMES! frames remaining in queue: 86
InitTimeBetweenFrames: 1.53316
SKIPPING 0 FRAMES! frames remaining in queue: 88
InitTimeBetweenFrames: 1.5665
SKIPPING 0 FRAMES! frames remaining in queue: 91
InitTimeBetweenFrames: 1.59983
SKIPPING 0 FRAMES! frames remaining in queue: 92
InitTimeBetweenFrames: 1.63316
SKIPPING 0 FRAMES! frames remaining in queue: 94
InitTimeBetweenFrames: 1.6665
SKIPPING 0 FRAMES! frames remaining in queue: 96
InitTimeBetweenFrames: 1.69983
SKIPPING 0 FRAMES! frames remaining in queue: 99
InitTimeBetweenFrames: 1.73316
SKIPPING 0 FRAMES! frames remaining in queue: 101
InitTimeBetweenFrames: 1.7665
SKIPPING 0 FRAMES! frames remaining in queue: 103
InitTimeBetweenFrames: 1.79983
SKIPPING 0 FRAMES! frames remaining in queue: 106
InitTimeBetweenFrames: 1.83316
SKIPPING 0 FRAMES! frames remaining in queue: 109
InitTimeBetweenFrames: 1.8665
SKIPPING 0 FRAMES! frames remaining in queue: 111
InitTimeBetweenFrames: 1.89983
SKIPPING 0 FRAMES! frames remaining in queue: 114
InitTimeBetweenFrames: 1.93316
SKIPPING 0 FRAMES! frames remaining in queue: 115
InitTimeBetweenFrames: 1.9665
SKIPPING 0 FRAMES! frames remaining in queue: 118
InitTimeBetweenFrames: 1.99983
SKIPPING 0 FRAMES! frames remaining in queue: 121
InitTimeBetweenFrames: 2.03316
SKIPPING 0 FRAMES! frames remaining in queue: 124
InitTimeBetweenFrames: 2.0665
SKIPPING 0 FRAMES! frames remaining in queue: 126
InitTimeBetweenFrames: 2.09983
SKIPPING 0 FRAMES! frames remaining in queue: 128
InitTimeBetweenFrames: 2.13316
SKIPPING 0 FRAMES! frames remaining in queue: 131
InitTimeBetweenFrames: 2.1665
SKIPPING 0 FRAMES! frames remaining in queue: 133
InitTimeBetweenFrames: 2.19983
SKIPPING 0 FRAMES! frames remaining in queue: 136
InitTimeBetweenFrames: 2.23316
SKIPPING 0 FRAMES! frames remaining in queue: 137
InitTimeBetweenFrames: 2.2665
SKIPPING 0 FRAMES! frames remaining in queue: 139
InitTimeBetweenFrames: 2.29983
SKIPPING 0 FRAMES! frames remaining in queue: 142
InitTimeBetweenFrames: 2.33316
SKIPPING 0 FRAMES! frames remaining in queue: 144
InitTimeBetweenFrames: 2.3665
SKIPPING 0 FRAMES! frames remaining in queue: 147
InitTimeBetweenFrames: 2.38983
SKIPPING 0 FRAMES! frames remaining in queue: 156
InitTimeBetweenFrames: 2.43316
SKIPPING 0 FRAMES! frames remaining in queue: 168
InitTimeBetweenFrames: 2.4665
SKIPPING 0 FRAMES! frames remaining in queue: 179
InitTimeBetweenFrames: 2.49983
SKIPPING 0 FRAMES! frames remaining in queue: 193
InitTimeBetweenFrames: 2.52316
SKIPPING 0 FRAMES! frames remaining in queue: 201
InitTimeBetweenFrames: 2.5665
SKIPPING 0 FRAMES! frames remaining in queue: 210
Scaling with rescaleFactor: 7.5105
Initialization: keep 9.8% (need 2000, have 20335)!
INITIALIZE FROM INITIALIZER (1971 pts)!
Frame history size: 79
Preparing keyframe: 78
Current mapping id: 78 create KF after: 78
Frames between KFs: 77
SPARSITY:  MinActDist 1.900000 (need 2000 points, have 1971 points)!
OPTIMIZE 1971 pts, 1971 active res, 0 lin res!
Initial Error           A(724061.624264)=(AV inf). Num: A('d) + M('d); ab 0.000000 0.000000!
Dynamic weight: 0
STEPS: A 0.0; B 0.0; R 0.0; T 0.0.      REJECT 0 (L -5.00, dir nan, ss 1.0):    A(2635841.448650)=(AV 20.332). Num: A('d) + M('d); ab 0.000000 0.000000!
Dynamic weight: 0.152984
STEPS: A 0.0; B 17.5; R 1.2; T 8.4.     ACCEPT 1 (L -3.00, dir 0.38, ss 1.0):   A(2621030.705718)=(AV 20.275). Num: A('d) + M('d); ab 0.000000 0.000000!
Dynamic weight: 0.155688
STEPS: A 0.0; B 1.4; R 0.7; T 0.2.      ACCEPT 2 (L -3.60, dir 0.78, ss 1.0):   A(2493161.072132)=(AV 15.377). Num: A('d) + M('d); ab 0.000000 0.000000!
Dynamic weight: 0.270667
STEPS: A 0.0; B 1.3; R 0.4; T 0.8.      ACCEPT 3 (L -4.20, dir 0.71, ss 1.0):   A(2464679.303179)=(AV 15.341). Num: A('d) + M('d); ab 0.000000 0.000000!
Dynamic weight: 0.271925
STEPS: A 0.0; B 1.6; R 0.6; T 0.3.      ACCEPT 4 (L -4.81, dir 0.92, ss 1.0):   A(2450703.405770)=(AV 15.304). Num: A('d) + M('d); ab 0.000000 0.000000!
Dynamic weight: 0.273267
STEPS: A 0.0; B 0.1; R 0.1; T 0.3.      REJECT 5 (L -5.00, dir 0.57, ss 1.0):   A(2451175.456030)=(AV 15.311). Num: A('d) + M('d); ab 0.000000 0.000000!
Num BA Iterations done: 6
LOG 78: 15.335 fine. Res: 1307 A, 0 L, 0 M; ('d / 'd) forceDrop. a=0.000000, b=0.000000. Window -883086680 (-309730729)
SKIPPING 0 FRAMES! frames remaining in queue: 225

And it crashes at "Pangolin-master\include\pangolin\gl\gl.hpp" on GlBufferData::Reinitialise(buffer_type, size_bytes, gluse, data);

Stepping back through the stack leads to:

https://github.com/lukasvst/dm-vio/blob/3b5319aa4f0cf03b59bcb7573ee66861de1681df/src/dso/IOWrapper/Pangolin/KeyFrameDisplay.cpp#L326

If I set:

bool disableAllDisplay = true;

It seems to run, and prints this:

Current mapping id: 795 create KF after: 597
SKIPPING 1 FRAMES! frames remaining in queue: 1124
lvl4, it -1 (l=0.010000 / 1.000000) INITIA: 0.000->52.568 (0 -> 293) (|inc| = 0.000000)!          0.00285573 -1.31823e-05   0.00260694   0.00733675    0.0349193  -0.00731295 AFF -0.0696036   -21.3145 (rel 0.932796    2.496)
lvl 4, it 0 (l=0.010000 / 1.000000) ACCEPT: 52.568->45.632 (293 -> 293) (|inc| = 0.010617)!     -0.00535996  0.00270789  0.00516661  0.00694484   0.0385329 -0.00598132 AFF -0.0870911   -17.6893 (rel 0.916626  5.70842)
lvl 4, it 1 (l=0.005000 / 1.000000) ACCEPT: 45.632->45.447 (293 -> 295) (|inc| = 0.002700)!     -0.00551905  0.00298596   0.0075905   0.0066162    0.038631  -0.0055161 AFF -0.0936622   -16.9519 (rel 0.910622  6.29255)
lvl 4, it 2 (l=0.002500 / 1.000000) ACCEPT: 45.447->44.646 (295 -> 299) (|inc| = 0.004170)!     -0.00880916  0.00414628  0.00959426  0.00624058   0.0396082 -0.00530238 AFF -0.0955588   -16.8275 (rel 0.908897  6.37298)
lvl 4, it 3 (l=0.001250 / 1.000000) ACCEPT: 44.646->44.455 (299 -> 299) (|inc| = 0.002194)!      -0.0107272  0.00500734
 0.00990702  0.00614703   0.0399294 -0.00511979 AFF -0.0932735   -17.1362 (rel 0.910976  6.11728)
lvl 4, it 4 (l=0.000625 / 1.124683) ACCEPT: 44.455->44.248 (299 -> 299) (|inc| = 0.001463)!      -0.0113372  0.00488776   0.0111308  0.00604992   0.0403728 -0.00530828 AFF -0.0922948   -17.2196 (rel 0.911868  6.05666)
lvl 4, it 5 (l=0.000312 / 1.337481) ACCEPT: 44.248->44.057 (299 -> 299) (|inc| = 0.002398)!      -0.0128475  0.00559371   0.0127964  0.00594494   0.0408012 -0.00528112 AFF -0.0925453   -17.1953 (rel 0.91164 6.07514)
lvl 4, it 6 (l=0.000156 / 1.590541) REJECT: 44.057->44.137 (299 -> 299) (|inc| = 0.002032)!      -0.0116611  0.00456846   0.0116854  0.00549699   0.0406654 -0.00555329 AFF -0.0952043   -16.9333 (rel 0.909219  6.27537)
lvl 4, it 7 (l=0.001000 / 1.000000) REJECT: 44.057->44.063 (299 -> 299) (|inc| = 0.001272)!      -0.0121042  0.00495188   0.0121016  0.00566451   0.0407167 -0.00545228 AFF -0.0942036   -17.0321 (rel 0.910129   6.1998)
lvl 4, it 8 (l=0.004000 / 1.000000) REJECT: 44.057->44.063 (299 -> 299) (|inc| = 0.001252)!      -0.0121135  0.00496145   0.0121143  0.00566873   0.0407198 -0.00545237 AFF -0.0941578   -17.0373 (rel 0.910171  6.19566)
lvl 4, it 9 (l=0.016000 / 1.000000) REJECT: 44.057->44.064 (299 -> 299) (|inc| = 0.001181)!      -0.0121473  0.00499659   0.0121611  0.00568454   0.0407311 -0.00545194 AFF -0.0939995   -17.0554 (rel 0.910315  6.18127)
lvl 4, it 10 (l=0.064000 / 1.000000) REJECT: 44.057->44.068 (299 -> 299) (|inc| = 0.000971)!     -0.0122487  0.00510065   0.0122994  0.00573444   0.0407614 -0.00544348 AFF -0.0936067    -17.101 (rel 0.910673  6.14478)
inc too small, break!

Is this a normal output?

Thank you!

lukasvst commented 1 year ago

The output with disableAllDisplay = turn; looks normal, if it's running like that for some time (until the dataset is finished).

Thanks for debugging that the error lies in the GUI. Unfortunately I'm not sure how much I will be able to help you with it, as I haven't run DM-VIO on Windows yet and don't plan to officially support it for now. But maybe you can get some information from this PR: https://github.com/lukasvst/dm-vio/pull/29

antithing commented 1 year ago

Thank you @lukasvst ! Using that PR has a TUM dataset (dataset-corridor1_512_16) running on Windows. However it is running very slowly (around 10Hz) on a high spec desktop machine.

I see that there is an option to use a realtime preset :

preset=0 # use 1 for realtime

But when i do that, i see:

Skipping comment line in IMU data.
IMU Id: 1520531829221612058
using pyramid levels 0 to 3. coarsest resolution: 64 x 64!
START PANGOLIN!
Switching to initializer state: RealtimeCoarseIMUInitState
PixelSelector: Using block sizes: 16, 16
LOADING ALL IMAGES!
OpenGL Error 502: GL_INVALID_OPERATION: The specified operation is not allowed in the current state.
In: D:\Testing\DM-VIO\dm-vio\thirdparty\Pangolin\include\pangolin/gl/gl.hpp, line 214

What can i do to solve this?

Thank you

antithing commented 1 year ago

More on speed.... Using: preset=2 nogui=1

Runs, and gives me:

5987 Frames (20.0 fps)
65.91ms per frame (single core);
infms per frame (multi core);
0.759x (single core);
0.000x (multi core);

Is that an expected speed? I am looking to hit 50 fps if I can.

Thanks!

lukasvst commented 1 year ago

Good to hear that is running with that PR, but it should definitely be running a lot faster. On a high-end machine I would expect at least 40fps (if your disk loads the images fast enough).

Can you upload the timings.txt here, which is generated in the results folder? In general, running it through a profiler might help to figure out why it's slow.

As to presets:

antithing commented 1 year ago

timings.txt usedSettingsdso.txt Thank you! I have run the sequnece using:

dmvio_dataset.exe files=D:/dataSet/dataset-corridor1_512_16/dso/cam0/images vignette=D:/dataSet/dataset-corridor1_512_16/dso/cam0/vignette.png imuFile=D:/dataSet/dataset-corridor1_512_16/dso/imu.txt gtFile=D:/dataSet/dataset-corridor1_512_16/dso/gt_imu.csv calib=D:/Testing/DM-VIO/dm-vio/configs/tumvi_calib/camera02.txt gamma=D:/Testing/DM-VIO/dm-vio/configs/tumvi_calib/pcalib.txt imuCalib=D:/Testing/DM-VIO/dm-vio/configs/tumvi_calib/camchain.yaml mode=0 use16Bit=1 preset=0 nogui=0 resultsPrefix=results/ settingsFile=D:/Testing/DM-VIO/dm-vio/configs/tumvi.yaml start=2 useimu=1

and I see:

======================
5987 Frames (20.0 fps)
126.60ms per frame (single core);
infms per frame (multi core);
0.395x (single core);
0.000x (multi core);
======================
[timings.txt](https://github.com/lukasvst/dm-vio/files/12159663/timings.txt)
[usedSettingsdso.txt](https://github.com/lukasvst/dm-vio/files/12159664/usedSettingsdso.txt)

Timings data attached. Thanks!

lukasvst commented 1 year ago

Thanks for sending the timings. It seems to be quite slow overall (i.e. not only in a particular part of the code). For reference: On my machine addActiveFrame takes 0.021s on average compared to your 0.11s and fullCoarseTracking takes 0.0028s, where your results show 0.068s.

Are you sure that you are building in RelWithDebInfo mode (and not in Debug)?