3DSSD Train 3 Classes

[RolandoAvides]

Hi!

I'm currently training 3DSSD in all 3 KITTI classes. I modified the configuration script by following similar 3d_3class scripts. However, I find odd these evaluation scores, specifically the cyclist and pedestrian AP.

I wonder if anyone has any thought about this. Besides, do you think it is wise to train this model in 3 classes? Or should I separate car/cyclist & ped?

Thanks in advance!

PS.: I can provide the configuration script if needed.

[RolandoAvides]

@Tai-Wang Thanks for adding to the community discussion. I think I might have found the problem though I can only check it on Monday.

[RolandoAvides]

Hello there!

Any update on this issue?

Thanks in advance

[yezhen17]

Hi @RolandoAvides ,

Sorry for the late reply. Can you provide the configuration script?

[RolandoAvides]

Hi @THU17cyz,

Thanks for your response. Yes, I'll provide it.

_base_ = [
    '../_base_/models/3dssd.py', 
    '../_base_/datasets/kitti-3d-3class.py',
    '../_base_/default_runtime.py'
]

# dataset settings
dataset_type = 'KittiDataset'
data_root = '/home/rmoreira/mmdetection3d/data/kitti/'
class_names = ['Car', 'Cyclist', 'Pedestrian']
point_cloud_range = [0, -40, -5, 70, 40, 3]
input_modality = dict(use_lidar=True, use_camera=False)
db_sampler = dict(
    data_root=data_root,
    info_path=data_root + 'kitti_dbinfos_train.pkl',
    rate=1.0,
    prepare=dict(filter_by_difficulty=[-1], filter_by_min_points=dict(Car=5, Pedestrian=10, Cyclist=10)),
    classes=class_names,
    sample_groups=dict(Car=12, Pedestrian=6, Cyclist=6))

file_client_args = dict(backend='disk')
# Uncomment the following if use ceph or other file clients.
# See https://mmcv.readthedocs.io/en/latest/api.html#mmcv.fileio.FileClient
# for more details.
# file_client_args = dict(
#     backend='petrel', path_mapping=dict(data='s3://kitti_data/'))

train_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='LIDAR',
        load_dim=4,
        use_dim=4,
        file_client_args=file_client_args),
    dict(
        type='LoadAnnotations3D',
        with_bbox_3d=True,
        with_label_3d=True,
        file_client_args=file_client_args),
    dict(type='PointsRangeFilter', point_cloud_range=point_cloud_range),
    dict(type='ObjectRangeFilter', point_cloud_range=point_cloud_range),
    dict(type='ObjectSample', db_sampler=db_sampler),
    dict(type='RandomFlip3D', flip_ratio_bev_horizontal=0.5),
    dict(
        type='ObjectNoise',
        num_try=100,
        translation_std=[1.0, 1.0, 0],
        global_rot_range=[0.0, 0.0],
        rot_range=[-1.0471975511965976, 1.0471975511965976]),
    dict(
        type='GlobalRotScaleTrans',
        rot_range=[-0.78539816, 0.78539816],
        scale_ratio_range=[0.9, 1.1]),
    dict(type='BackgroundPointsFilter', bbox_enlarge_range=(0.5, 2.0, 0.5)),
    dict(type='IndoorPointSample', num_points=16384),
    dict(type='DefaultFormatBundle3D', class_names=class_names),
    dict(type='Collect3D', keys=['points', 'gt_bboxes_3d', 'gt_labels_3d'])
]

test_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='LIDAR',
        load_dim=4,
        use_dim=4,
        file_client_args=file_client_args),
    dict(
        type='MultiScaleFlipAug3D',
        img_scale=(1333, 800),
        pts_scale_ratio=1,
        flip=False,
        transforms=[
            dict(
                type='GlobalRotScaleTrans',
                rot_range=[0, 0],
                scale_ratio_range=[1., 1.],
                translation_std=[0, 0, 0]),
            dict(type='RandomFlip3D'),
            dict(
                type='PointsRangeFilter', point_cloud_range=point_cloud_range),
            dict(type='IndoorPointSample', num_points=16384),
            dict(
                type='DefaultFormatBundle3D',
                class_names=class_names,
                with_label=False),
            dict(type='Collect3D', keys=['points'])
        ])
]

data = dict(
    samples_per_gpu=4,
    workers_per_gpu=4,
    train=dict(dataset=dict(pipeline=train_pipeline)),
    val=dict(pipeline=test_pipeline),
    test=dict(pipeline=test_pipeline))

evaluation = dict(interval=2)
# evaluation = 

# model settings
model = dict(
    bbox_head=dict(
        num_classes=3,
        bbox_coder=dict(
            type='AnchorFreeBBoxCoder', num_dir_bins=12, with_rot=True)))

# optimizer
lr = 0.002  # max learning rate
optimizer = dict(type='AdamW', lr=lr, weight_decay=0)
optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
lr_config = dict(policy='step', warmup=None, step=[80, 120])
# runtime settings
runner = dict(max_epochs=150, type='EpochBasedRunner')

# yapf:disable
log_config = dict(
    interval=30,
    hooks=[
        dict(type='TextLoggerHook'),
        dict(type='TensorboardLoggerHook')
    ])
# yapf:enable

I think one needs to modify the evaluation pipeline. However, I don't what are the needed configurations.

Thanks in advance!

[yezhen17]

Hi @RolandoAvides ,

Did you modify the code? For one thing, we do not have a semantic classification loss because the there is only one class. If you want to adapt it to 3 classes, at least you should supervise the classification? I'm not familiar with SSD3D so currently I'm not sure what you should further do.

[yezhen17]

If you are still interested in this problem, feel free to reopen the issue.

[zhanghm1995]

Hi, @RolandoAvides, I also need to train 3 classes by using 3DSSD network, I want to know did you have solved this problem? Thanks in advance.

[RolandoAvides]

Hi @zhanghming ,

Sorry for the late answer, but currently I'm quite busy.

I ended up training 3DSSD in all classes using a GitHub rep from Zye (based on OpenPCDet). I did not manage to implement this problem in MMDetection3D.

Thanks!

[zhanghm1995]

@RolandoAvides , Hi, Thanks for you reply. Is it the repo forked from OpenPCDet in your Github project? I didn't find any clues about 3DSSD there.

[RolandoAvides]

Hi @zhanghming I'll leave here the link to the fork I used: https://github.com/zye1996/OpenPCDet/tree/devel

My current fork does not have 3DSSD.

I hope it helps.

[jlqzzz]

@RolandoAvides @zhanghming Have you successfully implemented the multi-class version of 3dssd? Why is the accuracy so low on pedestrian and cyclist?

[RolandoAvides]

Hello @jlqzzz ,

I did not implemented the 3 classes using 3DSSD in this rep. You can check the linked I shared in my previous message in this issue.

The mAP is low in those classes because the original 3DSSD implemented in OpenPCDet does not support the three classes.

I hope I was able to help.

[jlqzzz]

Hello @jlqzzz ,

I did not implemented the 3 classes using 3DSSD in this rep. You can check the linked I shared in my previous message in this issue.

The mAP is low in those classes because the original 3DSSD implemented in OpenPCDet does not support the three classes.

I hope I was able to help.

@RolandoAvides

Yes, my above results are based on the openpcdet you mentioned.

But I don't understand why the accuracy of pedestrian and cyclist is so low ! Is the parameter setting of a part of the code wrong?

What should I modify to achieve satisfactory results on these two categories?

[RolandoAvides]

I suspect the model is not prepared to detect all 3 classes. It solely detects cars.

[wangy12]

I had the same low mAP values for pedestrians and cyclists when training for 3 classes. I tested and visualized my trained model and found that pedestrians and cyclists are detected. The problem is in testing/validate config. I modified the '3dssd.py' in the 'model' folder as follows. This will resolve the problem. test_cfg=dict( nms_cfg=dict(type='nms', iou_thr=0.1), sample_mod='spec', score_thr=0.2, per_class_proposal=False, # Change from True to False max_output_num=60))