search

xdspacelab / openvslam

OpenVSLAM: A Versatile Visual SLAM Framework
https://openvslam.readthedocs.io/
2.97k stars 870 forks source link

Stereo fisheye data, rectification, tum vi #410

Open dinarkino opened 4 years ago

dinarkino commented 4 years ago

Hi!

I have several questions related to using openvslam with stereo fisheye data. I was trying to run it on TUM VI dataset and OpenLORIS dataset. There is a config file in Examples folder for the TUM VI. Using the same file as an example I created a config for the openloris. So, here are the questions:

  1. First of all, it seems like openvslam easily lose its trajectory right after leaving the starting room in tum vi dataset. I think the reason for that is very fast moves inside the corridor. If we reset a map after corridor then it's again working but then again LOST. Is that normal behaviour for slam on this dataset? Also, it seems like the image is very like "fluid", maybe this is typical for the rectified fish-eye image 1 2

  2. Secondly, I don't fully understand which parameters should we use and how to obtain them if there are no rectification matrices. Now it seems like there is a possibility to use both rectified images and to rectify them during the run (but if we will delete Stereo Rectification params then we will see an error at the beginning). At the same time, I tried to use OpenCV's function cv2.fisheye.stereoRectify(k1, d1, k2, d2, imageSize, R, tvec, flags=cv2.CALIB_ZERO_DISPARITY) and got camera matrix for the left image (fx, fy, cx, cy) and two R matrixes. Then I run the algorithm and got a very bad image. So, the question is what is the proper way to get these parameters?

From this

this

  1. And the last one. I found an interesting thing. If we will replace values of fx, fy, cx, cy by the values from the original camera matrix, then image became not fluidic at all, smaller and looks more like monocular. Then actually I was able to run both tum-vi (with no LOST at the corridor, but anyway after one minute it was again LOST) and openloris (here trajectory is pretty good). I don't have much experience in the area, so I'm curious what happens in that case and why it works better because seems like I'm just giving wrong parameters to the algorithm as it's said in the config file: 
# new "rectified" matrices is the first three cols of the projection matrix which is calculated with cv::stereoRectify()
# e.g. fx = P1[0][0] or P2[0][0], cx = P1[0][2] or P2[0][2]
#      fy = P1[1][1] or P2[1][1], cy = P1[1][2] or P2[1][2]

photo_2020-07-27_18-27-47

HaibingJin commented 4 years ago

I'm new to OpenVSLAM. Could you please tell me how to write the config file of stereo camera?

oym1994 commented 3 years ago

Hi, I am also using the OpenLORIS dataset, could you please teach me how to get the config parameters? Thank you in advance!