Ground Truth in Kitti-360 Vs Ground Truth in Kitti

[Jesser1855]

Hello,

I am working on multi-sensor fusion and 3d detection algorithms for my master thesis in TUM.

In the normal kitti data-set kitti data-se: :

• image_2
• velodyne,
• calibration files
• label_2 (ground truth per frame) are provided.

The issue is that I need a dataset (image and lidar data) taken in sequence and not like the ones provided by the normal kitti dataset. I found out that the kitti-360 is the kitti dataset that I need.

MY question here, does the kitti-360 provides calibration and ground truth like the ones provided in normal kitti dataset ?

this is the format of the label_2 (ground truth per frame )files in the normal kitti [kitti ground truth as label_2.](https://github.com/bostondiditeam/kitti/blob/master/resources/devkit_object/readme.txt)

Values Name Description

---

1 type Describes the type of object: 'Car', 'Van', 'Truck', 'Pedestrian', 'Person_sitting', 'Cyclist', 'Tram', 'Misc' or 'DontCare' 1 truncated Float from 0 (non-truncated) to 1 (truncated), where truncated refers to the object leaving image boundaries 1 occluded Integer (0,1,2,3) indicating occlusion state: 0 = fully visible, 1 = partly occluded 2 = largely occluded, 3 = unknown 1 alpha Observation angle of object, ranging [-pi..pi] 4 bbox 2D bounding box of object in the image (0-based index): contains left, top, right, bottom pixel coordinates 3 dimensions 3D object dimensions: height, width, length (in meters) 3 location 3D object location x,y,z in camera coordinates (in meters) 1 rotation_y Rotation ry around Y-axis in camera coordinates [-pi..pi] 1 score Only for results: Float, indicating confidence in detection, needed for p/r curves, higher is better.

---

       **If it does not deliver such label_2 files directly, Is there a way to build such files from other files provided by the kitti-360 ?** 

       Thank you in advance 
       best regards 
       Jesser Bouzid 

[JohnalDsouza]

Im facing the same problem. the documentation of kitti360 for 3d bounding boxes isnt very good. I am trying to visualize the 3d bounding boxes in BEV along with the lidar points. but due too lack of documentation, its a real struggle

[yiyiliao]

Hi, one student of our team is working on providing a script for this. We will add it to the devkit once it is ready.

[abhi1kumar]

Hi @JohnalDsouza You could check out the converted KITTI-360 in KITTI labels in our SeaBird repo KITTI-360 section. It contains processed files in the KITTI format.

[NaveenPrasaad]

Hi, one student of our team is working on providing a script for this. We will add it to the devkit once it is ready.

Hello @yiyiliao, We are working on a model that takes KITTI format for BEV generation. Is the script for converting KITTI 360 to KITTI dataset available so that we can convert and use it for our model?

Hello @abhi1kumar, Actually we tried using the processed files in the KITTI format from SeaBird repo, but as our model needs the velo_to_cam coordinates in the calibration file, we got an error and hence we couldn't use it. Additionally we need road labels in label file to get the drivable area which is also not available in the SeaBird repo.

[abhi1kumar]

Hi @NaveenPrasaad

our model needs the velo_to_cam coordinates in the calibration file

Yes, the SeaBird repo does not have velo_to_cam information since the SeaBird models are image-based and do not use lidar data. However, you can do this on your end:

Here is a brief summary of what KITTI-360 files contain

├── calibration/intrinsics.txt       # rect_to_intri, cam_to_rect
├── calibration/cam_to_pose.txt      # ego_to_camera
├── calibration/cam_to_velo.txt      # cam_to_velo
├── data_poses/drive_name/poses.txt  # world_to_ego

world_to_rect       = np.matmul(np.matmul(cam_to_rect, ego_to_camera), world_to_ego)
camera_calib_final  = np.matmul(rect_to_intri, world_to_rect)
world_to_vel        = np.matmul(cam_to_velo, camera_calib_final)

Additionally we need road labels in label file to get the drivable area which is also not available in the SeaBird repo.

I did not fully understand your question. If you are looking for BEV semantic and panoptic segmentation labels for the drivable area (roads, lanes), you should use the KITTI-360 PanopticBEV dataset, which provides these labels for the KITTI-360 dataset. Follow the SeaBird instructions to get these labels for the KITTI-360 images.

[HarisAmir]

Hi, one student of our team is working on providing a script for this. We will add it to the devkit once it is ready.

Hello @yiyiliao 👋

Any update on this end? I was also interested in converting the labels from KITTI-360 to KITTI format