search

SpectacularAI / HybVIO

HybVIO visual-inertial odometry and SLAM system
https://arxiv.org/abs/2106.11857
GNU General Public License v3.0
459 stars 92 forks source link

Low overlap stereo cameras #22

Open mateosss opened 2 years ago

mateosss commented 2 years ago

Hello, I have a pair of stereo cameras from a VR headset that point at different angles. I've been able to get HybVIO to track stereo features only with -useRectification, but even then it doesn't seem possible for HybVIO to track features on the non-overlapping areas of the images.

  1. Is there a way to avoid having to prerectify the images in this case?
  2. Is it possible to track non-overlapping features in HybVIO?

Below is a stereo frame example obtained by running this command:

./main -i=../data/benchmark/ody-easy -p -c -s -windowResolution=640 -useSlam -useStereo -displayStereoMatching -useRectification

Over this HybVIO-formatted dataset. The original EuRoC-formatted dataset can be found here.

2022-06-14-135018_1284x521_scrot

oseiskar commented 2 years ago

Hello! Your findings are consistent with our understanding of the capabilities of the current HybVIO code

  1. Sometimes rectification is needed in practice or the left-to-right LK tracking fails (even though it "should not"). The current rectification implementation is very slow, which could be worked around by rectifying manually with, e.g, OpenCV before feeding the data to HybVIO (this also needs to be reflected in the calibration in parameters.txt). There are currently no work-arounds for the need for rectification in this case, except fixing/modifying the stereo LK-tracking implementation
  2. The current code does not have the capability of tracking non-overlapping features in the case of -useStereo. It should be possible to extend the algorithm to support them
mateosss commented 2 years ago

I see, thank you!