ORB_SLAM2

[koko19]

Thank you very much for this code.

I am trying to use TUM VI-dataset to run ORB-SLAM2 in stereo mode. However, in order to do so I had to make changes to the code to ensure new cv::fisheye model is used for image rectification. The problem is that I cannot estimate good undistortion parameters using these functions. First I run

  cv::size imgSize(512,512)
  cv::fisheye::stereoRectify(K_l, D_l,
                             K_r, D_r,
                             s_size, R, T,
                             R_l, R_r, P_l, P_r,
                             Q, cv::CALIB_ZERO_DISPARITY, imgSize, 0.0, 1.0); 

To get projection and rectification matrices R_l,r and P_l,r using data in camchain yaml file. R and T are extracted from T_cn_cnm1:.Then I use these matrices in cv::fisheye::initUndistortRectifyMap to get map matrices used by ORBSLAM2. However, acquired P_l,r matrices seem wrong and when applied to undistort the image, complete black image is received. The matrices are:

P_l [49.23458128237724, 0, 2406.333490668249, 0; 0, 49.23458128237724, -103.7019266270119, 0; 0, 0, 1, 0] P_r [49.23458128237724, 0, 2406.333490668249, -4.976989132152498; 0, 49.23458128237724, -103.7019266270119, 0; 0, 0, 1, 0]

I have also modified your script to get these matrices and same result is achieved. I have tested the code with opencv sample matrices and in that case when different K and D are used results are fine using the same code. Also, ORBSLAM2 works fine like this using other datasets.

Do you have any idea what I am doing wrong?

[HTLife]

If your fisheye model code works fine on opencv sample K, D, I suspect that TUM-VI gave a wrong K,D.

To verify this guess, we need to calibrate TUM-VI by ourself.

[NikolausDemmel]

I assume you are Kruno, so I'm responding here instead of via email.

I don't know about opencv stereo undistortion. I suggest to

• Start with undistortion of the two images independently (like implemented in this repo). Verify that the result looks good (images have straight lines).
• Verify that you use the correct transformation for the camera extrinsics. A typical error is to use the inverse instead.

We provide calibration datasets, so feel free to redo the calibration yourself, if you want. You can also look at the Kalibr reports we provide for our calibration to check that this is ok for you. However, the calibration we provide should be not too bad, since with that we can run both OKVIS and ROVIO successfully.

If you do notice an issue with the calibration, please let us know. Thanks!

[dymymao]

I assume you are Kruno, so I'm responding here instead of via email.

I don't know about opencv stereo undistortion. I suggest to

• Start with undistortion of the two images independently (like implemented in this repo). Verify that the result looks good (images have straight lines).
• Verify that you use the correct transformation for the camera extrinsics. A typical error is to use the inverse instead.

We provide calibration datasets, so feel free to redo the calibration yourself, if you want. You can also look at the Kalibr reports we provide for our calibration to check that this is ok for you. However, the calibration we provide should be not too bad, since with that we can run both OKVIS and ROVIO successfully.

If you do notice an issue with the calibration, please let us know. Thanks!

---

Thanks for TUM_VI datasets. but I still don't know how to use the fisheye images,since I want to get the undistort stereo rectify images for ORB-SLAM2.

[NikolausDemmel]

Thanks for TUM_VI datasets. but I still don't know how to use the fisheye images,since I want to get the undistort stereo rectify images for ORB-SLAM2.

Well, I'm not sure what response you expect here? If you want, follow my advice above, check the documentation of cv::fisheye::stereoRectify and Kalibr's calibration format.

I believe Kalibr's "equidistant" camera model is the same as opencv's fisheye model, which is supported by the fact that the single-image rectification looks ok (see Readme).

If you find a concrete problem with our calibration, please let us know.

[sheroy666]

I assume you are Kruno, so I'm responding here instead of via email. I don't know about opencv stereo undistortion. I suggest to

• Start with undistortion of the two images independently (like implemented in this repo). Verify that the result looks good (images have straight lines).
• Verify that you use the correct transformation for the camera extrinsics. A typical error is to use the inverse instead.

We provide calibration datasets, so feel free to redo the calibration yourself, if you want. You can also look at the Kalibr reports we provide for our calibration to check that this is ok for you. However, the calibration we provide should be not too bad, since with that we can run both OKVIS and ROVIO successfully. If you do notice an issue with the calibration, please let us know. Thanks!

Thanks for TUM_VI datasets. but I still don't know how to use the fisheye images,since I want to get the undistort stereo rectify images for ORB-SLAM2.

Hi! I wonder if you have fix your problem , i am trying to run ORB_SLAM 2 using TUM VI-dataset but have faced the same problem.