Will RDIoU work with pointpillars?

[ruiniao]

Hi,

I tried simply switch AnchorHeadSingle out with AnchorHeadRDIoU_3CAT in pointpillar.yaml. But the training session gave eval metrics with all 0s. What's the correct configuration for the replacement? Thanks!

[hlsheng1]

Yes, it works for pointpillar. Maybe u are facing the nan issue. U can check the output values to see whethere they are nans.

[Dhagash4]

Can you maybe elaborate more on what to modify, as I tried changing it to pointpillars and it is giving me share errors? P.S. I am using custom dataset and RDIoU is working fine and I want to try different representations before base backbone

CLASS_NAMES:  ['Car','Truck','Pedestrian']

DATA_CONFIG: 
    _BASE_CONFIG_: path/to/dataset/config 

MODEL:
    NAME: PointPillar

    VFE:
        NAME: PillarVFE
        WITH_DISTANCE: False
        USE_ABSLOTE_XYZ: True
        USE_NORM: True
        NUM_FILTERS: [64]

    MAP_TO_BEV:
        NAME: PointPillarScatter
        NUM_BEV_FEATURES: 64

    BACKBONE_2D:
        NAME: CTBEVBackbone_3CAT

        LAYER_NUMS: [2, 0]
        LAYER_STRIDES: [1, 2]
        NUM_FILTERS: [128, 256]
        UPSAMPLE_STRIDES: [1, 2]
        NUM_UPSAMPLE_FILTERS: [256, 256]

    DENSE_HEAD:
        NAME: AnchorHeadRDIoU_3CAT
        CLASS_AGNOSTIC: False

        USE_DIRECTION_CLASSIFIER: True
        DIR_OFFSET: 0.78539
        DIR_LIMIT_OFFSET: 0.0
        NUM_DIR_BINS: 2

        ANCHOR_GENERATOR_CONFIG: [
            {
                'class_name': 'Car',
                'anchor_sizes': [[3.9, 1.6, 1.56]],
                'anchor_rotations': [0, 1.57],
                'anchor_bottom_heights': [-1.78],
                'align_center': False,
                'feature_map_stride': 4,
                'matched_threshold': 0.6,
                'unmatched_threshold': 0.45
            },
            {
                'class_name': 'Truck',
                'anchor_sizes': [[12.0, 2.85, 4.0]],
                'anchor_rotations': [0, 1.57],
                'anchor_bottom_heights': [-0.6],
                'align_center': False,
                'feature_map_stride': 4,
                'matched_threshold': 0.5,
                'unmatched_threshold': 0.35
            },
            {
                'class_name': 'Pedestrian',
                'anchor_sizes': [[0.8, 0.6, 1.73]],
                'anchor_rotations': [0, 1.57],
                'anchor_bottom_heights': [-0.6],
                'align_center': False,
                'feature_map_stride': 2,
                'matched_threshold': 0.5,
                'unmatched_threshold': 0.35
            }
        ]

        TARGET_ASSIGNER_CONFIG:
            NAME: AxisAlignedTargetAssigner
            POS_FRACTION: -1.0
            SAMPLE_SIZE: 512
            NORM_BY_NUM_EXAMPLES: False
            MATCH_HEIGHT: False
            BOX_CODER: ResidualCoder

        LOSS_CONFIG:
            LOSS_WEIGHTS: {
                'cls_weight': 1.0,
                'loc_weight': 2.0,
                'dir_weight': 0.2
            }

    POST_PROCESSING:
        RECALL_THRESH_LIST: [0.3, 0.5, 0.7]
        SCORE_THRESH: 0.3
        OUTPUT_RAW_SCORE: False

        EVAL_METRIC: kitti

        NMS_CONFIG:
            MULTI_CLASSES_NMS: False
            NMS_TYPE: nms_gpu
            NMS_THRESH: 0.1
            NMS_PRE_MAXSIZE: 4096
            NMS_POST_MAXSIZE: 500

OPTIMIZATION:
    BATCH_SIZE_PER_GPU: 2
    NUM_EPOCHS: 80

    OPTIMIZER: adam_onecycle
    LR: 0.003
    WEIGHT_DECAY: 0.01
    MOMENTUM: 0.9

    MOMS: [0.95, 0.85]
    PCT_START: 0.4
    DIV_FACTOR: 10
    DECAY_STEP_LIST: [35, 45]
    LR_DECAY: 0.1
    LR_CLIP: 0.0000001

    LR_WARMUP: False
    WARMUP_EPOCH: 1

    GRAD_NORM_CLIP: 10

and i get following error in axis_aligned_target_assigner.py

shape '[1, 250, 250, -1]' is invalid for input of size 31250

[hlsheng1]

We use axis_aligned_target_assigner_add_gt.py instead of axis_aligned_target_assigner.py. Have u ever changed the code? Or could u provide more details?

[Dhagash4]

No I didnt change the code and you are right the error is in axis_aligned_target_assigner_add_gt.py, so what I did is copied pointpillar.py from OpenPCDet repo and changed the config to the one mentioned above, also providing more information about some of the parameters POINT_CLOUD_RANGE = [0, -40, -2.5, 80, 40, 2.5] and voxel_size=[016,0.16,5] and PC_RANGE is same when I am training with rdiou_3cat.yaml but voxel_size=[0.05, 0.056818182, 0.12195122]

This is the error coming:

Traceback (most recent call last):                                                                                                                                                        | 0/4522 [00:00<?, ?it/s]
  File "tools/train.py", line 204, in <module>
    main()
  File "tools/train.py", line 159, in main
    train_model(
  File "/home/ded2st/dev/RDIoU/tools/train_utils/train_utils.py", line 86, in train_model
    accumulated_iter = train_one_epoch(
  File "/home/ded2st/dev/RDIoU/tools/train_utils/train_utils.py", line 38, in train_one_epoch
    loss, tb_dict, disp_dict = model_func(model, batch)
  File "/home/ded2st/dev/RDIoU/pcdet/models/__init__.py", line 30, in model_func
    ret_dict, tb_dict, disp_dict = model(batch_dict)
  File "/home/ded2st/.conda/envs/rdiou/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ded2st/dev/RDIoU/pcdet/models/detectors/pointpillar.py", line 11, in forward
    batch_dict = cur_module(batch_dict)
  File "/home/ded2st/.conda/envs/rdiou/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ded2st/dev/RDIoU/pcdet/models/dense_heads/anchor_head_rdiou_3cat.py", line 344, in forward
    targets_dict = self.assign_targets(
  File "/home/ded2st/dev/RDIoU/pcdet/models/dense_heads/anchor_head_template.py", line 96, in assign_targets
    targets_dict = self.target_assigner.assign_targets(
  File "/home/ded2st/dev/RDIoU/pcdet/models/dense_heads/target_assigner/axis_aligned_target_assigner_add_gt.py", line 126, in assign_targets
    'box_cls_labels': [t['box_cls_labels'].view(*feature_map_size, -1) for t in target_list],
  File "/home/ded2st/dev/RDIoU/pcdet/models/dense_heads/target_assigner/axis_aligned_target_assigner_add_gt.py", line 126, in <listcomp>
    'box_cls_labels': [t['box_cls_labels'].view(*feature_map_size, -1) for t in target_list],
RuntimeError: shape '[1, 250, 250, -1]' is invalid for input of size 31250

[hlsheng1]

This is not the problem of RDIoU and I guess u will meet the same issue when u just run original Pointpillar with ur config. The problem is that the shape of 'box_cls_labels' is equal to the anchors initialization. Your feature_map_size here is 250250, but ur anchor setting is 31250=125125*2. (The anchors setting should be same as feature_map_size). So, perhaps you can check the difference here.

[Dhagash4]

Thanks for fast response was able to modify the config and resolve the issue. One more question can we use AnchorHeadRDIoU_3CAT for more than 3 categories?

[hlsheng1]

Sure, just make few change in get_clsreg_targets() function like the previous version, it should be ok.

[LiangziWang9]

Hello, I also encountered the same problem, how did you solve it? Looking forward for your reply, thank you.@Dhagash4

'box_cls_labels': [t['box_cls_labels'].view(*feature_map_size, -1) for t in target_list], RuntimeError: shape '[1, 200, 544, -1]' is invalid for input of size 54400

[Dhagash4]

Hi, I solved the issue long back, so I do not remember it clearly. But check the anchor setting, particularly which feature map stride you are passing. As @hlsheng1 mentions anchors setting should be the same as feature_map_size. I hope this helps

[LiangziWang9]

Thank you! :) @Dhagash4