how to understand the rec_to_lidar function?

[rockywind]

def rect_to_lidar(self, pts_rect): """ :param pts_lidar: (N, 3) :return pts_rect: (N, 3) """ pts_rect_hom = self.cart_to_hom(pts_rect) # (N, 4) R0_ext = np.hstack((self.R0, np.zeros((3, 1), dtype=np.float32))) # (3, 4) R0_ext = np.vstack((R0_ext, np.zeros((1, 4), dtype=np.float32))) # (4, 4) R0_ext[3, 3] = 1 V2C_ext = np.vstack((self.V2C, np.zeros((1, 4), dtype=np.float32))) # (4, 4) V2C_ext[3, 3] = 1

    pts_lidar = np.dot(pts_rect_hom, np.linalg.inv(np.dot(R0_ext, V2C_ext).T))
    return pts_lidar[:, 0:3]`

`

[rockywind]

I think the CaDDN's output is based on lidar coordination rather than camera coordination.

[codyreading]

That's correct, the rec_to_lidar function transforms points in the Rectified Camera Frame to the LiDAR Frame. The output of CaDDN is in the LiDAR frame, and then gets converted back to the Camera frame for KITTI evaluation.

[rockywind]

Thanks for your help. How to generate the R0_rect matrix. The R0_rect_matrix is very similar to the identity matrix.

To project a point from Velodyne coordinates into the left color image, you can use this formula: x = P2 R0_rect Tr_velo_to_cam y For the right color image: x = P3 R0_rect Tr_velo_to_cam y

[codyreading]

In the code you mentioned that would be self.R0