weiyithu / SurroundOcc

[ICCV 2023] SurroundOcc: Multi-camera 3D Occupancy Prediction for Autonomous Driving
Apache License 2.0
812 stars 104 forks source link

process_your_own_data.py中对动态物体重构时,为什么z轴要提高一半的高度呢? #65

Open yzhh95 opened 1 year ago

yzhh95 commented 1 year ago
    gt_bbox_3d = dict['gt_bbox_3d']
    locs = gt_bbox_3d[:,0:3]
    dims = gt_bbox_3d[:,3:6]
    rots = gt_bbox_3d[:,6:7]
    gt_bbox_3d[:, 2] += dims[:, 2] / 2.     

    ################## bbox placement ##############
    object_points_list = []
    object_semantic_list = []
    for j, object_token in enumerate(dict['object_tokens']):
        for k, object_token_in_zoo in enumerate(object_token_zoo):
            if object_token==object_token_in_zoo:
                points = object_points_vertice[k]
                Rot = Rotation.from_euler('z', rots[j], degrees=False)
                rotated_object_points = Rot.apply(points)
                points = rotated_object_points + locs[j]     # 前面gt_bbox改变了,这个点云的重投影回去的位置也会改变啊,
Xpangz commented 1 year ago

老哥,你有根据他给的数据和代码生成稠密的标签吗,我生成的标签比较稀疏,不知道为何?

lqzhao commented 1 year ago

@yzhh95 这么做是因为 points_in_boxes_cpu 这个函数要求 输入的 xyz 是bbox中心坐标,但是nuscenes数据的xyz是bbox底面中心的坐标,因此需要手动加上一半的高度才能满足函数要求

qihao-plus commented 1 year ago

https://github.com/weiyithu/SurroundOcc/blob/main/tools/generate_occupancy_with_own_data/process_your_own_data.py#L160C73-L165 如果 “points_in_boxes_cpu 这个函数要求 输入的 xyz 是bbox中心坐标”, 那为什么这里不需要 + height 呢?

boxes = np.load(os.path.join(bbox_path, 'bbox{}.npy'.format(i)))
object_category = np.load(os.path.join(bbox_path, 'object_category{}.npy'.format(i)))
boxes_token = np.load(os.path.join(bbox_path, 'boxes_token{}.npy'.format(i)))

points_in_boxes = points_in_boxes_cpu(torch.from_numpy(pc0[:, :3][np.newaxis, :, :]),
                                              torch.from_numpy(boxes[np.newaxis, :]))
qihao-plus commented 1 year ago

其次是为什么要做:

object_points = object_points[:,:3] - dict['gt_bbox_3d'][i][:3] # why
rots = dict['gt_bbox_3d'][i][6]
Rot = Rotation.from_euler('z', -rots, degrees=False)
rotated_object_points = Rot.apply(object_points)
lqzhao commented 1 year ago

其次是为什么要做:

object_points = object_points[:,:3] - dict['gt_bbox_3d'][i][:3] # why
rots = dict['gt_bbox_3d'][i][6]
Rot = Rotation.from_euler('z', -rots, degrees=False)
rotated_object_points = Rot.apply(object_points)

是为了将 拼接好的物体点云,按照当前帧的bbox的方向,再次放回到场景中。因此在前面遍历所有帧时,会将每个残缺物体的方向都记录下来,存在dict['gt_bbox_3d']中

lqzhao commented 1 year ago

其次是为什么要做:

object_points = object_points[:,:3] - dict['gt_bbox_3d'][i][:3] # why
rots = dict['gt_bbox_3d'][i][6]
Rot = Rotation.from_euler('z', -rots, degrees=False)
rotated_object_points = Rot.apply(object_points)

如果你的数据标注的xyz就是物体中心的话,你可以删去 gt_bbox_3d[:, 2] += dims[:, 2] / 2.

qihao-plus commented 1 year ago

多谢回复,我的疑惑是,在代码中,为什么

https://github.com/weiyithu/SurroundOcc/blob/main/tools/generate_occupancy_with_own_data/process_your_own_data.py#L155-L165 不需要对齐中心,而 https://github.com/weiyithu/SurroundOcc/blob/main/tools/generate_occupancy_with_own_data/process_your_own_data.py#L286-L302 这里需要

lqzhao commented 1 year ago

多谢回复,我的疑惑是,在代码中,为什么

https://github.com/weiyithu/SurroundOcc/blob/main/tools/generate_occupancy_with_own_data/process_your_own_data.py#L155-L165 不需要对齐中心,而 https://github.com/weiyithu/SurroundOcc/blob/main/tools/generate_occupancy_with_own_data/process_your_own_data.py#L286-L302 这里需要

谢谢你的提醒,这两处应该保持一致,我将注释掉第二处对齐中心的代码

qihao-plus commented 1 year ago

嗯嗯,如果 template GT bbox 已经是对齐中心的,可以注释掉第二处对齐中心。 如果 template GT bbox 是如你所说,和 nuscenes 一致,那么应该是第一步 += dims[:, 2] / 2. ?

lqzhao commented 1 year ago

是的

artofstate commented 1 year ago

@lqzhao 您好,看到您说points_in_boxes_cpu要求输入的 xyz 是bbox中心坐标?但是我看到源码中说 需要的是底部坐标?那是不是说明应该先gt_bbox_3d[:, 2] -= dims[:, 2] / 2,然后后边恢复gt_bbox_3d[:, 2] += dims[:, 2] / 2

SteveJMao commented 9 months ago

@lqzhao 您好,看到您说points_in_boxes_cpu要求输入的 xyz 是bbox中心坐标?但是我看到源码中说 需要的是底部坐标?那是不是说明应该先gt_bbox_3d[:, 2] -= dims[:, 2] / 2,然后后边恢复gt_bbox_3d[:, 2] += dims[:, 2] / 2

你好,我查看源码后也发现是要求的底部坐标,这里应该遵循哪一个pipline呢

SteveJMao commented 9 months ago

作者您好!查阅mmcv文档以及测试后 image 如果是bbox中心坐标确实应该先用gt_bbox_3d[:, 2] -= dims[:, 2] / 2转换到bbox的底面再调用points_in_boxes_cpu