How to use the principal points of image intrinsics in RansacLib

[zhangpj]

Hello, first of all thank you for sharing this project. I am trying to use RansacLib in visual localization, and I test the result of localization in aachen dataset. However the result is not satisfactory. I notice that in camera_pose_estimation.cc , the focal_length information is used in the function:

CalibratedAbsolutePoseEstimator::PixelsToViewingRays(
      focal_length, focal_length, *points2D, rays);

but I didn't find any use of the principal points information. I modify the function PixelsToViewingRays like this:

void CalibratedAbsolutePoseEstimator::PixelsToViewingRays(
    const double focal_x, const double focal_y, const double c_x, const double c_y, const Points2D& points2D,
    ViewingRays* rays) {
  const int kNumData = static_cast<int>(points2D.size());
  // Creates the bearing vectors and points for the OpenGV adapter.
  rays->resize(kNumData);
  for (int i = 0; i < kNumData; ++i) {
    (*rays)[i] = points2D[i].homogeneous(); 
    (*rays)[i][0] = ((*rays)[i][0]-c_x)/focal_x;
    (*rays)[i][1] = ((*rays)[i][1]-c_y)/focal_y;
    (*rays)[i].normalize(); 
  }

But I still can't get the correct result. Can you tell me how to use the information of the principal points of image intrinsics in RansacLib, or just discard it like you do in localization.cc? Thanks in advance!

[tsattler]

@zhangpengju0316 The transformation looks correct. The problem is most likely in line 127. Due to my choice of the coordinate system in which my matching code operates (Bundler's coordinate system rather than Colmap's coordinate system that is used for the ground truth poses), I mirror the y- and z-coordinate of the 3D points. You probably want to remove this.

The pose estimation code provided in the examples assumes that the 2D keypoints are already centered around the principal point and have been undistorted:

// The input to the constructor are the camera focal lengths f_x, f_y, a set
  // of 2D keypoint positions, and the corresponding 3D points. The 2D points
  // are expected to be centered around the principal point (i.e., the
  // principal has already been subtracted) and to be undistorted.

Note that the code provided in examples/ is meant to illustrate how to use the code. I very much recommend verifying whether it fits your application scenario before using it.

[zhangpj]

@tsattler Thanks for your reply, I fixed my problem! According to your suggestion, the 2D points should be centered around the principal point. So the 2D point cannot just be centered around the principal point in PixelsToViewingRays , because some other places also have used points2D. So I subtract the principal before the 2D points are input into RansacLib module, it works!!! Thank you again.