jj-gomez
commented
3 years ago I would need more information to figure out correctly you issue. When you say that ORB-SLAM3 detects a small number of features , do you mean that almost no FAST corner are extracted or that FAST corners are correctly extracted but they are not matched in consecutive images?
Our stereo fisheye does not require rectification but requires a few extra parameters in the calibration file. I am posting here another answer I wrote related with those parameters in another issue (#88 ):
@ankuPRK @XiongLianga This will solved in a new realease this week. However, I will provide you with a short explanation:
Stereo cameras are usually rectified for the ease of finding stereo matches. This rectifying step is very well-known for pinhole cameras and it has negligible effects on the FoV of the original images.
However, rectifying stereo fisheye cameras yields to a trade back depending of the calibration of the rectified images:
- Keep the characteristic FoV of the fisheye cameras but greatly magnify objects at the boundaries of the image and loose resolution at the center of the image.
- Loose a huge quantity of FoV but make things in the center of the image look fine (like a pinhole camera).
These reasons discourage the use of rectification over fisheye images. Instead of that, we use the 2 raw stereo images modifying in a new stereo approch. In this approach, we consider the stereo rig as 2 monocular cameras constrianed by:
- A constant relative SE(3) transformation.
- Optionally an image region that observe the same portion of the scene (the lapping area).
Within this lapping area, we can get stereo, truescaled, points which combined with the relative pose of the cameras allows us to retrieve the real scale of the map. This lapping area is likely to be different depending of your stereo rig so we decided to include it as a parameter to identify in which regions of the image should consider features as "stereo" or "monocular".
This approach allows us to use all the features extracted in both images which renders a more robust stereo system. For example, if one camera gets completely occluded, the other one can be used for a monocular tracking in the meanwhile the other occluded.
Since the code is in a beta version and we ran a lot of experiments, there is still a lot of dead code and hardcoded parameters (in this case, the lapping area). That is why I apologize for the confusion. However, as said before, this will solved shortly with a parameter parser and a correct implementation of the lapping area.
Regarding ROS stereo+IMU, it should work if provided the correct calibration.
It would be also helpful some example images in which you are having troubles to process.
longzhenren
malleshamdasari
ryleymcc
hanb257
I am using the Intel RealSense T265 camera. I have calibrated it well by Kalibr. But I found the ORB-SLAM3 did not fitted with this camera, it detected little features in the image. So that it always tracked failure as following. Does fisheye image need stereo rectification? Does default code in the ros_stereo_inertial OK with the fisheye camera? Do I need do other operation to fit this camera?