search

SpectacularAI / HybVIO

HybVIO visual-inertial odometry and SLAM system
https://arxiv.org/abs/2106.11857
GNU General Public License v3.0
450 stars 92 forks source link

HybVIO assume/tuned for parallel stereo cameras? #45

Closed ArmandB closed 10 months ago

ArmandB commented 10 months ago

I've been using HybVIO on the recently released Monado VR dataset: https://huggingface.co/datasets/collabora/monado-slam-datasets and have been observing a lot of failures, even after increasing the parameters like BA problem size and pose history size in order to account for the higher fps (54).

In many cases, the number of tracks is extremely low (many new key points are detected in the left frame but very few are tracked to the right frame), but tracking proceeds much better when I enable the "useRectification" flag. When I contacted the dataset's author, they said that when they tuned Basalt to run on it, they had to modify the algorithm because it was tuned for parallel stereo cameras and the Valve Index (the VR headset the data is from) cameras are canted.

Just wanted to ask and see if this might be what's causing the problem here as well.

Thank you in advance!

oseiskar commented 10 months ago

Hi Armand. The HybVIO codebase would need numerous changes to work for real-time inside-out tracking in an actual headset. Supporting non-parallel stereo cameras (which is currently not properly supported, like you suspected) is just one of them.

These features are included in our commercial SDK, which we recommend for all real-time embedded use cases.

oseiskar commented 10 months ago

Also already discussed in https://github.com/SpectacularAI/HybVIO/issues/22

ArmandB commented 10 months ago

Dear Otto, Sorry for the duplicate question and thank you so much for the speedy response! I had looked at #22 about a year ago and incorrectly remembered that it was only about HybVIO not supporting key points in the non-overlapped camera FoV.

Thank you again for your help!