misaligned of gt_3d_BBox on the scene

[ammaryasirnaich]

What is the feature?

I am trying to plot 3D ground truth boxes, and I have followed the demo code in Drawing 3D Boxes on Point Cloud. In this code have tried to plot the 3D boxes from the 00000.8.pkl file by using the below code

bboxes_3d = []

for instance in info_file['data_list'][0]['instances']:
    bboxes_3d.append(instance['bbox_3d'])

# Define a single RGB color for all bounding boxes (e.g., green)
single_bbox_color = [0, 255, 0]

# Duplicate the color for all bounding boxes
bbox_colors = [single_bbox_color] * len(bboxes_3d)

points = np.fromfile('demo/data/kitti/000008.bin', dtype=np.float32)
points = points.reshape(-1, 4)
visualizer = Det3DLocalVisualizer()
# set point cloud in visualizer
visualizer.set_points(points,vis_mode='add')

# Draw and visualize each bounding box with the duplicated color
for bbox, color in zip(bboxes_3d, bbox_colors):
    visualizer.draw_bboxes_3d(LiDARInstance3DBoxes(torch.tensor(bbox).unsqueeze(0)), bbox_color=np.array([color]))
visualizer.show()

but I get a misalignment boxes

help from someone will be appreciated

Any other context?

No response

[szy4017]

bboxes_3d

I met the same problem, do you fix it now?

[ammaryasirnaich]

@szy4017 i did manage to solve it, had to transform the raw 3dbox information to the CameraInstance3DBoxes instance

# convert gt_bboxes_3d to velodyne coordinates with `lidar2cam`
lidar2cam = np.array(info_file['data_list'][0]['images']['CAM2']['lidar2cam'])

gt_instances_3d.bboxes_3d = CameraInstance3DBoxes(bbox3d).convert_to(Box3DMode.LIDAR,
                                                 np.linalg.inv(lidar2cam))
gt_instances_3d.labels_3d = labels_3d

[szy4017]

@szy4017 i did manage to solve it, had to transform the raw 3dbox information to the CameraInstance3DBoxes instance

# convert gt_bboxes_3d to velodyne coordinates with `lidar2cam`
lidar2cam = np.array(info_file['data_list'][0]['images']['CAM2']['lidar2cam'])

gt_instances_3d.bboxes_3d = CameraInstance3DBoxes(bbox3d).convert_to(Box3DMode.LIDAR,
                                                 np.linalg.inv(lidar2cam))
gt_instances_3d.labels_3d = labels_3d

Thanks for your reply. I found another solution to fix this bug. I tried to convert the CameraInstance3DBoxes to LiDARInstance3DBoxes directly, did not require for lidar2cam.

from mmdet3d.structures.bbox_3d import Box3DMode
bbox3d_cam = CameraInstance3DBoxes(bbox3d_cam)
bbox3d_lidar = Box3DMode.convert(bbox3d_cam, Box3DMode.CAM, Box3DMode.LIDAR)