Odometry evaluation on KITTI

[jinyummiao]

Thanks for your excellent job! Could you please provide a evaluation script to test the odometry performance on KITTI dataset like nuscenes and waymo?

[YihongSun]

Thank you for your interest in our work!

As far as I am aware, evaluating odometry on KITTI requires a separate split. (I have not worked with KITTI in a while, so I might be mistaken.)

Since odometry results are available for datasets with more independently moving objects (main focus of our work), implementing odometry eval on KITTI is too involved for this project.

Nevertheless, if you process the odometry annotations similar to nuScenes and Waymo, the current eval script should work out-of-the-box.

Hope this helps and feel free to reply if you have any other questions or concerns!

[jinyummiao]

Thanks for your quick reply! Maybe i can evaluate the odometry performance as monodepth2?

[jinyummiao]

I have another question about the pose estimation. In eval/odometry.py line 81, the local poses between frame t-1 and frame t are calculated by 'np.linalg.inv(np.dot(np.linalg.inv(gt_global_poses[i - 1]), gt_global_poses[i]))', but in my opinion, 'np.dot(np.linalg.inv(gt_global_poses[i - 1]), gt_global_poses[i])' is already the relative pose between frame t-1 and frame t, which corresponds to the predicted results of the pose decoder. So, why do we need to inverse the pose matrix again?

[YihongSun]

Hi, the odometry eval script is mainly adopted from Monodepth2, where the same line is also found here.

My initial thought would be that an additional inv would change the direction of the relative pose (can be shown after distributing the inverse into the matmul), and the median scaling would remove any directional mismatch (via a negative sign).

I would need to run the code to verify, and I will get back to you!

[YihongSun]

Hi, sorry for the delayed response! I can confirm that the median scaling would remove the directional mismatch.