search

UZ-SLAMLab / ORB_SLAM3

ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
GNU General Public License v3.0
6.64k stars 2.57k forks source link

Non-overlapping fisheye stereo: Is it possible? #240

Open martinakos opened 3 years ago

martinakos commented 3 years ago

I’ve managed to run orbslam3 with stereo fisheye when there is overlapping between both camera FoVs. Now I’ve tried to use orbslam3 with two back to back fisheye cameras and there is no overlapping between the both cameras FoVs. However, it can’t find any features in the current frame and the map is not initialised.

As I understand from this issue #88 the fisheye stereo mode works by having two independent monocular fisheye cameras. The features that are in the overlapping region are considered stereo features and the features outside the overlapping region are mono features. So in my case there would be only mono features.

I wonder if the problem is with the initialisation expecting some stereo features to initialise the scale of the map, but as there are only mono features it can't initialise. I see that for monocular fisheye there is an initialisation step where features are tracked until there is enough parallax and then the corresponding landmarks are introduced in the map. As the stereo mode doesn’t use this initialisation that may be preventing this non-overlaping configuration from working. Could this be the reason?

In my configuration file I set the lapping parameters both begin and end to 0 to indicate that there is no overlapping. And set the Tlr transform so that the cameras are facing back to back.

tejalbarnwal commented 3 years ago

@martinakos did you find a solution to using two cameras with no overlapping features?

martinakos commented 3 years ago

Nope. I think it would need a monocular initialisation but as it's configured for stereo it won't initialise.

tejalbarnwal commented 3 years ago

@martinakos any way you know, I could use through which I can calculate the overlapping area between images of stereo pair, given the intrinsic and extrinsic of the cameras?