Axis definitions

[bmankirLinker]

I'm trying to transform the dataset to KITTI format, but I think the export_to_KITTI file is not updated so I've made some modifications but the axis transformation part is more complicated. Are the axis definition is the same as in NuScenes, such that for KITTI transformation we can apply kitti_to_nu_lidar = Quaternion(axis=(0, 0, 1), angle=np.pi / 2) ?

[bmankirLinker]

For anyone who's interested in transformation, can update the line as below if wanna convert KITTI format. kitti_to_nu_lidar = Quaternion(axis=(0, 0, 1), angle=np.pi)

[gakkiri]

Thank you so much for pointing this out. I am also doing the kitti format conversion. Is there anything else I should pay attention to?
I replaced most of the self.nusc class in export_kitti.py with class provided by lyft. Does this work?

Edit: I found that the center provided by get-box in the two SDK is different, so I replaced it.

[Tai-Wang]

So do your experiments show that the LiDAR axis definition in lyft data is "x-back, y-right, z-up"? @bmankirLinker

[bmankirLinker]

That transformation definition makes the https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/scripts/export_kitti.py#L153 assertion pass. However after converting the PCD and GTs accordingly, GT and the image don't match. And also training with PRCNN also gives bad results, unlike NuScenes. So apparently I'm missing some stuff as well but unfortunately, hope an update will come.

[gakkiri]

Oh, I can comment an assertion without modifying the axis definition, which seems to get the correct visualization, use the render module in the script. @bmankirLinker

[Tai-Wang]

Maybe some functions are not affected by its axis definition, but we are indeed encountered bugs when inferencing orientations of objects by using some models adapted from NuScenes/KITTI. BTW, @joinssmith do you know the clear definition of LiDAR in Lyft dataset?

[gakkiri]

@Tai-Wang Sorry, I don't know.

[bmankirLinker]

@joinssmith so you've used the same transformation as in NuScenes, but also updated z of cuboid GT based on definition?

[Tai-Wang]

All right, hope an update...Thanks all the same~

[pyaf]

Just found out:

KITTI lidar frame is 90 degrees rotated from nuScenes lidar frame

here So, kitti_to_nu_lidar = Quaternion(axis=(0, 0, 1), angle=np.pi / 2) makes sense.

[pyaf]

I can confirm two things:

1. kitti_to_nu_lidar = Quaternion(axis=(0, 0, 1), angle=np.pi / 2) and with no assert (velo_to_cam_rot.round(0) == np.array([[0, -1, 0], [0, 0, -1], [1, 0, 0]])).all() gives perfectly transformed boxes in KITTI format but 90 degrees rotated x-y axes.

2. kitti_to_nu_lidar = Quaternion(axis=(0, 0, 1), angle=np.pi) and with assert (velo_to_cam_rot.round(0) == np.array([[0, -1, 0], [0, 0, -1], [1, 0, 0]])).all()gives perfectly aligned axes, but 90 degrees horizontally-off boxes.

[pyaf]

@joinssmith

Edit: I found that the center provided by get-box in the two SDK is different, so I replaced it.

Could you please tell me what exactly is different in them?

[gakkiri]

@pyaf Sorry for my belated clarification. I ignored the transformation of the coordinate system at that time, and in fact the BOX given is the same. I'm sorry very much.

[pyaf]

@joinssmith there's nothing to be sorry about :)

The thing is if we could swap x and y axes, we'd be able to fix this issue.