NVIDIA-AI-IOT / Lidar_AI_Solution

A project demonstrating Lidar related AI solutions, including three GPU accelerated Lidar/camera DL networks (PointPillars, CenterPoint, BEVFusion) and the related libs (cuPCL, 3D SparseConvolution, YUV2RGB, cuOSD,).
Other
1.27k stars 224 forks source link

CenterPoint model #247

Open super-liuyang opened 4 months ago

super-liuyang commented 4 months ago

Is there a pre-converted ONNX model available for the CenterPoint .pth model

hopef commented 3 months ago

We used the official pth model in CenterPoint merely. You can find the URL from here.

mx2013713828 commented 3 months ago

We used the official pth model in CenterPoint merely. You can find the URL from here.

Hi,thanks your work. I want to train my custom centerpoint model, can I use openpcdet impl ? and can i export the model that trained in openpcdet codebase? waiting for your reply~!

hopef commented 3 months ago

But openpcdet is a pointpillars-based architecture. This is contradictory to the sparse conv we support. Are you sure you want to train a pointpillars-based model?

mx2013713828 commented 3 months ago

But openpcdet is a pointpillars-based architecture. This is contradictory to the sparse conv we support. Are you sure you want to train a pointpillars-based model?

But I found the centerpoint config in openpcdet like below, I think its voxel based model, can i use it ? ` MODEL: NAME: CenterPoint

VFE:
    NAME: DynPillarVFE
    WITH_DISTANCE: False
    USE_ABSLOTE_XYZ: True
    USE_NORM: True
    NUM_FILTERS: [ 64, 64 ]

MAP_TO_BEV:
    NAME: PointPillarScatter
    NUM_BEV_FEATURES: 64

BACKBONE_2D:
    NAME: BaseBEVBackbone
    LAYER_NUMS: [3, 5, 5]
    LAYER_STRIDES: [2, 2, 2]
    NUM_FILTERS: [64, 128, 256]
    UPSAMPLE_STRIDES: [0.5, 1, 2]
    NUM_UPSAMPLE_FILTERS: [128, 128, 128]

DENSE_HEAD:
    NAME: CenterHead
    CLASS_AGNOSTIC: False

    CLASS_NAMES_EACH_HEAD: [
        ['car'], 
        ['truck', 'construction_vehicle'],
        ['bus', 'trailer'],
        ['barrier'],
        ['motorcycle', 'bicycle'],
        ['pedestrian', 'traffic_cone'],
    ]

    SHARED_CONV_CHANNEL: 64
    USE_BIAS_BEFORE_NORM: True
    NUM_HM_CONV: 2
    SEPARATE_HEAD_CFG:
        HEAD_ORDER: ['center', 'center_z', 'dim', 'rot', 'vel']
        HEAD_DICT: {
            'center': {'out_channels': 2, 'num_conv': 2},
            'center_z': {'out_channels': 1, 'num_conv': 2},
            'dim': {'out_channels': 3, 'num_conv': 2},
            'rot': {'out_channels': 2, 'num_conv': 2},
            'vel': {'out_channels': 2, 'num_conv': 2},
        }

    TARGET_ASSIGNER_CONFIG:
        FEATURE_MAP_STRIDE: 4
        NUM_MAX_OBJS: 500
        GAUSSIAN_OVERLAP: 0.1
        MIN_RADIUS: 2

    LOSS_CONFIG:
        LOSS_WEIGHTS: {
            'cls_weight': 1.0,
            'loc_weight': 0.25,
            'code_weights': [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.2, 0.2, 1.0, 1.0]
        }

    POST_PROCESSING:
        SCORE_THRESH: 0.1
        POST_CENTER_LIMIT_RANGE: [-61.2, -61.2, -10.0, 61.2, 61.2, 10.0]
        MAX_OBJ_PER_SAMPLE: 500
        NMS_CONFIG:
            NMS_TYPE: nms_gpu
            NMS_THRESH: 0.2
            NMS_PRE_MAXSIZE: 1000
            NMS_POST_MAXSIZE: 83

POST_PROCESSING:
    RECALL_THRESH_LIST: [0.3, 0.5, 0.7]

    EVAL_METRIC: kitti

`

hopef commented 3 months ago

Yeah, this configuration is pointpillars based centerpoint model. It will be supported by TensorRT.

mx2013713828 commented 3 months ago

Yeah, this configuration is pointpillars based centerpoint model. It will be supported by TensorRT.

well, I dont know how to export pytorch model to TensorRT model, can I just use export-scn.py?

hopef commented 3 months ago

No, you can't use the export-scn.py. This code may be helpful to you.

mx2013713828 commented 3 months ago

No, you can't use the export-scn.py. This code may be helpful to you.

Thanks, I got it ,when i use pillar based model ,I can use cuda-pointpillars export code, but I also need to export the center head ,right?