Question TensorRT deployment

Hi @Haiyang-W ,

Thank you for sharing the deployment script for trt! I have successfully trained a model with PCDET and generated an ONNX file. However, I encountered a problem when using trtexec to generate the engine file.export_dsvt_dynamic.log

ckpt file onnx config.yaml

According to my own data, I modified the dimensions of the input and output like this. Are there any questions?

I hope for your help! Looking forward to your reply!

[01/29/2024-18:08:40] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: ShapeHostToDevice Tactic: 0
[01/29/2024-18:08:40] [V] [TRT] =============== Computing costs for 
[01/29/2024-18:08:40] [V] [TRT] *************** Autotuning format combination: Int32(E0,36,1), Float((* 384 (# 0 (SHAPE src))),(* 192 (# 0 (SHAPE src))),192,1), Int32(E1,36,1), Bool(E0,36,1), Bool(E1,36,1), Float(192,1), Int32(), Int32(), Int32(), Int32(), Int32() -> Float(192,1) where E0=(* 36 (# 1 (SHAPE set_voxel_inds_tensor_shift_0))) E1=(* 36 (# 1 (SHAPE set_voxel_inds_tensor_shift_1))) ***************
[01/29/2024-18:08:40] [V] [TRT] --------------- Timing Runner: {ForeignNode[Slice_72...Add_1851]} (Myelin)
[01/29/2024-18:08:40] [W] [TRT] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: 
[01/29/2024-18:08:40] [W] [TRT]  (# 1 (SHAPE set_voxel_inds_tensor_shift_1))
[01/29/2024-18:08:40] [W] [TRT]  (# 0 (SHAPE src))
[01/29/2024-18:08:40] [W] [TRT]  (# 1 (SHAPE set_voxel_inds_tensor_shift_0))
[01/29/2024-18:09:10] [E] Error[1]: Unexpected exception 
[01/29/2024-18:09:10] [E] Error[2]: [builder.cpp::buildSerializedNetwork::561] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[01/29/2024-18:09:10] [E] Engine could not be created from network
[01/29/2024-18:09:10] [E] Building engine failed
[01/29/2024-18:09:11] [E] Failed to create engine from model.
[01/29/2024-18:09:11] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8200] # trtexec --onnx=/model_dsvt_0109/model_dsvt_0109_dynamic.onnx --saveEngine=/model_dsvt_0109/model_dsvt_0109_dynamic.engine --workspace=4096 --verbose --buildOnly --device=0 --tacticSources=+CUDNN,+CUBLAS,-CUBLAS_LT --minShapes=src:10000x192,set_voxel_inds_tensor_shift_0:2x1000x36,set_voxel_inds_tensor_shift_1:2x800x36,set_voxel_masks_tensor_shift_0:2x1000x36,set_voxel_masks_tensor_shift_1:2x800x36,pos_embed_tensor:4x2x10000x192 --optShapes=src:50000x192,set_voxel_inds_tensor_shift_0:2x4500x36,set_voxel_inds_tensor_shift_1:2x4000x36,set_voxel_masks_tensor_shift_0:2x4500x36,set_voxel_masks_tensor_shift_1:2x4000x36,pos_embed_tensor:4x2x50000x192 --maxShapes=src:110000x192,set_voxel_inds_tensor_shift_0:2x5500x36,set_voxel_inds_tensor_shift_1:2x5000x36,set_voxel_masks_tensor_shift_0:2x5500x36,set_voxel_masks_tensor_shift_1:2x5000x36,pos_embed_tensor:4x2x110000x192


        input_shapes = {
            "src": {
                "min_shape": [10000, 192],
                "opt_shape": [50000, 192],
                "max_shape": [110000, 192],
            },
            "set_voxel_inds_tensor_shift_0": {
                "min_shape": [2, 1000, 36],
                "opt_shape": [2, 4500, 36],
                "max_shape": [2, 5500, 36],
            },
            "set_voxel_inds_tensor_shift_1": {
                "min_shape": [2, 800, 36],
                "opt_shape": [2, 3500, 36],
                "max_shape": [2, 4500, 36],
            },
            "set_voxel_masks_tensor_shift_0": {
                "min_shape": [2, 1000, 36],
                "opt_shape": [2, 4500, 36],
                "max_shape": [2, 5500, 36],
            },
            "set_voxel_masks_tensor_shift_1": {
                "min_shape": [2, 800, 36],
                "opt_shape": [2, 3500, 36],
                "max_shape": [2, 4500, 36],
            },
            "pos_embed_tensor": {
                "min_shape": [4, 2, 10000, 192],
                "opt_shape": [4, 2, 50000, 192],
                "max_shape": [4, 2, 110000, 192],
            },

model config like this:


DATA_CONFIG:
    _BASE_CONFIG_: tools/cfgs/dataset_configs/ownwz_dataset_5c.yaml
    # SAMPLED_INTERVAL: {'train': 1, 'test': 1}
    POINT_CLOUD_RANGE: [-51.2, -25.6, -4.0, 51.2, 25.6, 2.0]
    DATA_AUGMENTOR:
         DISABLE_AUG_LIST: ['placeholder']


    VFE:
        NAME: DynamicVoxelVFE
        WITH_DISTANCE: False
        USE_ABSLOTE_XYZ: True
        USE_NORM: True
        NUM_FILTERS: [ 192, 192 ]

    BACKBONE_3D:
        NAME: DSVT
        INPUT_LAYER:
            sparse_shape: [512, 256, 1] #grid size
            downsample_stride: []
            d_model: [192]  # input token dim, len mean stage num
            set_info: [[36, 4]] # max num of voxel per set, block num per stage
            window_shape: [[12, 12, 1]] # win num
            hybrid_factor: [2, 2, 1] # x, y, z hybird win param
            shifts_list: [[[0, 0, 0], [6, 6, 0]]] # shift stride
            normalize_pos: False

        block_name: ['DSVTBlock']
        set_info: [[36, 4]]
        d_model: [192]
        nhead: [8]
        dim_feedforward: [384]
        dropout: 0.0
        activation: gelu
        output_shape: [512, 256]
        conv_out_channel: 192
        # You can enable torch.utils.checkpoint to save GPU memory
        # USE_CHECKPOINT: True

    MAP_TO_BEV:
        NAME: PointPillarScatter3d
        INPUT_SHAPE: [512, 256, 1]
        NUM_BEV_FEATURES: 192

    BACKBONE_2D:
        NAME: BaseBEVResBackbone
        LAYER_NUMS: [ 1, 2, 2 ]
        LAYER_STRIDES: [ 1, 2, 2 ]
        NUM_FILTERS: [ 128, 128, 256 ]
        UPSAMPLE_STRIDES: [ 1, 2, 4 ]
        NUM_UPSAMPLE_FILTERS: [ 128, 128, 128 ]

Haiyang-W / DSVT

Question TensorRT deployment #68