bad calib camera matrix

[DiTo97]

I recorded a posed RGB-D dataset with RTAB-map with a T265 + D435i dual setup, but the exported calib files seem off.

In particular, camera matrices are not valid intrinsic matrices. I will share an example; why is that?

camera_name: "3348"
image_width: 640
image_height: 480
camera_matrix:
   rows: 3
   cols: 3
   data: [ 6.0879376220703125e+02, 0., 3.1840899658203125e+02, 0., 0.,
       6.0877636718750000e+02, 2.4450122070312500e+02, 0., 0. ]
distortion_coefficients:
   rows: 1
   cols: 5
   data: [ 0., 0., 0., 0., 0. ]
distortion_model: plumb_bob
rectification_matrix:
   rows: 3
   cols: 3
   data: [ 1., 0., 0., 0., 1., 0., 0., 0., 1. ]
projection_matrix:
   rows: 3
   cols: 4
   data: [ 6.0879376220703125e+02, 0., 3.1840899658203125e+02, 0., 0.,
       6.0877636718750000e+02, 2.4450122070312500e+02, 0., 0., 0., 1.,
       0. ]
local_transform:
   rows: 3
   cols: 4
   data: [ 1.39685639e-03, 2.05031354e-02, 9.99788821e-01,
       8.42269696e-03, -9.99853790e-01, 1.70659740e-02, 1.04696734e-03,
       3.56464945e-02, -1.70409046e-02, -9.99644101e-01, 2.05239765e-02,
       2.72104051e-02 ]

On a side note, what's the meaning of local transform?

[matlabbe]

That was a 9 years bug, thx to point out! Should be fixed in commit above. A workaround is to recover the camera matrix from the projection matrix (left 3x3 part of the matrix).

For the local_transform, it is the transform between the base frame of the robot and the camera (e..g, including optical rotation).

[DiTo97]

Thanks @matlabbe for the prompt intervention,

one more off-topic question. What axis convention does RTAB-map follows, if any?

[matlabbe]

RTAB-Map follows REP103 and REP105, the local_transform is the transform between base_link (x->forward, y->left, z->up) and camera_optical_frame (x->right, y->down, z->forward).