Evaluating GLOMAP on ETH3D

[mattiadurso]

Hi, I am trying to reproduce the experiments done on ETH3D MVS DSLR (table 3 in the paper) but I am not finding the code use use to compute the AUC, either in the repo or IMC23 code.

It it possible to add it to the repo or link it?

[lpanaf]

Hi, thanks for the interest. For the AUC score, we use code from Pixel-Perfect Structure-from-Motion at https://github.com/cvg/pixel-perfect-sfm/blob/40f7c1339328b2a0c7cf71f76623fb848e0c0357/pixsfm/eval/eth3d/localization.py#L138. You would need to write your own functions for estimating pair-wise relative pose error, but the principle is to iterate through all $n^2$ pairs

[mattiadurso]

Hi, thank for the answer! I linked the points (1,2,3) with comments in the code (#1,#2,#3).

Considering the following function in the linked script

def compute_pose_error(image_gt: pycolmap.Image, pose_dict: Dict[str, Any]): #1
    if pose_dict['success']:
        R_w2c_gt = image_gt.rotmat()
        t_c2w_gt = image_gt.projection_center() #2
        image = pycolmap.Image(
            "dummy", qvec=pose_dict['qvec'], tvec=pose_dict['tvec'])
        R_w2c = image.rotmat()
        t_c2w = image.projection_center() #2 

        dt = np.linalg.norm(t_c2w_gt - t_c2w) #3
        cos = np.clip(((np.trace(R_w2c_gt @ R_w2c.T)) - 1) / 2, -1, 1)
        dR = np.rad2deg(np.abs(np.arccos(cos)))
    else:
        dt, dR = np.inf, 180
    return dt, dR

• 1 when I run

pose_dict = {
    'qvec': np.array([0.0, 0.0, 0.0, 1.0]),
    'tvec': np.array([0.0, 0.0, 0.0]),
}

pycolmap.Image("dummy", qvec=pose_dict['qvec'], tvec=pose_dict['tvec'])

I get

TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. pycolmap.Image()
    2. pycolmap.Image(name: str = '', points2D: pycolmap.ListPoint2D = [], cam_from_world: pycolmap.Rigid3d = Rigid3d(quat_xyzw=[0, 0, 0, 1], t=[0, 0, 0]), camera_id: int = 4294967295, id: int = 4294967295)
    3. pycolmap.Image(name: str = '', keypoints: numpy.ndarray[numpy.float64[m, 2]] = array([], shape=(0, 2), dtype=float64), cam_from_world: pycolmap.Rigid3d = Rigid3d(quat_xyzw=[0, 0, 0, 1], t=[0, 0, 0]), camera_id: int = 4294967295, id: int = 4294967295)
    4. pycolmap.Image(arg0: dict)
    5. pycolmap.Image(**kwargs)

Invoked with: 'dummy'; kwargs: qvec=array([0., 0., 0., 1.]), tvec=array([0., 0., 0.])

What should i pass to it?

• 2 What does this function? I looked for it but I didn't find the code even in the c++ implementation. May you provide the formula?

• 3 In the paper are reported angular measurements (e.g. AUC @ 1°) but it seems here a metric error is computed.

[lpanaf]

Can you maybe check out #67 for the concrete formulation? We did not use this code for comparison: for ETH3D we use relative pose error, and the code you post here indeed compares the absolute camera center error

[mattiadurso]

Can you maybe check out https://github.com/colmap/glomap/issues/67 for the concrete formulation?

Yes. Thank for the help. I close this issue.