open-mmlab / mmdeploy

OpenMMLab Model Deployment Framework
https://mmdeploy.readthedocs.io/en/latest/
Apache License 2.0
2.77k stars 636 forks source link

Parsing CenterPoint Output ONNX Nodes #1079

Closed OrcunCanDeniz closed 2 years ago

OrcunCanDeniz commented 2 years ago

I managed to get the ONNX export for mmdet3d's centerpoint. However I am not sure how do I parse the outputs. For example, I have the ONNX model as in the screenshot, for a network with 3 tasks each with 1 class. Direction, 3D boxes and confidences from all tasks are concated at the output. How do i be sure that any given index or slice of these output tensors is associated to a specific task t and class c. What is the recommended approach to getting this kind of mapping info for each architecture, is it simply the same order in the cfg file maybe?

Screenshot from 2022-09-20 11-41-24

  pts_bbox_head=dict(
    type='CenterHead',
    in_channels=sum([128, 128, 128]),
    tasks=[
        dict(num_class=1, class_names=['car']),
        dict(num_class=1, class_names=['bicycle']),
        dict(num_class=1, class_names=['pedestrian']),
    ],
    common_heads=dict(
        reg=(2, 2), height=(1, 2), dim=(3, 2), rot=(2, 2)),

     ...
tpoisonooo commented 2 years ago
  1. The output (dir_score, bbox_preds and scores) needs postprocess here: https://github.com/open-mmlab/mmdeploy/blob/1c5dcb799badbad717f6bfed615acb4a8e52f92f/mmdeploy/codebase/mmdet3d/models/centerpoint.py#L68

centerpoint__get_bbox is actually called here

Finally get_bbox() would return real bbox/score/label. The label value [0,1,2] maps to [car, bicyle, ped]

  1. Furthermore, End2End centerpoint model (without postprocess) is still WIP.
OrcunCanDeniz commented 2 years ago

Thanks !