About nuscenes open dataset

[starnstar]

Hello,

Thanks for your great work! I have a question about the unified coordinate system:

In "LIDAR" coordinate system of mmdetection3d, the relative coordinate of bottom center in a LiDAR box is (0.5, 0.5, 0):

but there is no translation for the nuscenes dataset. please see in the mmdet3d/datasets/nuscenes_dataset.py 165 line - "TODO":

        # the nuscenes box center is [0.5, 0.5, 0.5], we change it to be
        # the same as KITTI (0.5, 0.5, 0)
        # TODO: Unify the coordinates
        if self.load_type in ['fov_image_based', 'mv_image_based']:
            gt_bboxes_3d = CameraInstance3DBoxes(
                ann_info['gt_bboxes_3d'],
                box_dim=ann_info['gt_bboxes_3d'].shape[-1],
                origin=(0.5, 0.5, 0.5))
        else:
            gt_bboxes_3d = LiDARInstance3DBoxes(
                ann_info['gt_bboxes_3d'],
                box_dim=ann_info['gt_bboxes_3d'].shape[-1],
                origin=(0.5, 0.5, 0.5)).convert_to(self.box_mode_3d)

        ann_info['gt_bboxes_3d'] = gt_bboxes_3d

So I wonder if there are truly some difference between KITTI/WAYMO and NUSCENES? Looking forward to your reply, thanks a lot！

[JingweiZhang12]

@starnstar. Hi, we have set the origin as (0.5,0.5,0.5) in NuScenes to fit the original box coordinates in NuScenes. Then, we can just use the interface of the class LiDARInstance3DBoxes, such as 'center', 'corners', 'height' and so on, ignoring the inner complex translation.

[starnstar]

Thanks for your reply! Can I understand like this, as long as the boxes and the point clouds can be aligned, there is no need to set the box center at the bottom? And will you integrate BEVFusion model into mmdetection3d? Thanks a lot!

[starnstar]

Sorry, I just saw that you have implemented BEVFusion in the lastest version v1.1.0. Thanks for your great work! Is this part just write for inference, not for training? https://github.com/open-mmlab/mmdetection3d/tree/main/projects/BEVFusion

I still have some questions about nuscenes dataset. In https://github.com/open-mmlab/mmdetection3d/blob/main/mmdet3d/datasets/nuscenes_dataset.py#L172, the nuscenes box's origin is (0.5, 0.5, 0.5). Then in https://github.com/open-mmlab/mmdetection3d/blob/4ff136163ee4e6436e7f5c44f73c4a4932aa7657/mmdet3d/structures/bbox_3d/base_box3d.py#L68 you changed it to (0.5, 0.5, 0), right?

[shliu0]

Hello, I'm a beginner, could someone tell me

1. why there are three versions of bbox bottom center coordinates in mmdet3d?
   • (0.5, 0.5, 0.5) mmdet3d/datasets/nuscenes_dataset.py parse_ann_info 🔗
   • (0.5, 1.0, 0.5) in mmdet3d/structures/bbox_3d/box_3d_mode.py 🔗
   • (0.5, 0.5, 0) in mmdet3d/structures/bbox_3d/base_box3d.py 🔗
2. what is the intrinsic relationship between them, and what kind of coordinate order should I follow when implementing my own data processing pipeline?