[Bug] How to get the pred_instances for the mask2Former tensorrt model

[moliwei2019]

Checklist

• [x] I have searched related issues but cannot get the expected help.
• [x] 2. I have read the FAQ documentation but cannot get the expected help.
• [x] 3. The bug has not been fixed in the latest version.

Describe the bug

I want to convert the mask2Former model to tensorrt for instance segmentation tasks. Currently, I have completed the conversion and obtained cls_logits and mask_logits. How can I obtain the instance segmentation results. I tried to set export_postprocess_mask to True, but it didn't seem to work.

"codebase_config": { "type": "mmdet", "task": "ObjectDetection", "model_type": "panoptic_end2end", "post_processing": { "export_postprocess_mask": true, "score_threshold": 0.8, "iou_threshold": 0.5, "max_output_boxes_per_class": 200, "pre_top_k": 5000, "keep_top_k": 100, "background_label_id": -1 } }, "onnx_config": { "type": "onnx", "export_params": true, "keep_initializers_as_inputs": false, "opset_version": 13, "save_file": "end2end.onnx", "input_names": [ "input" ], "output_names": [ "cls_logits", "mask_logits" ], "input_shape": [ 960, 960 ], "optimize": true },

I'm a bit confused about how to get this return result from the TRT model. I hope technical experts can help me solve my questions

Reproduction

python3 /root/workspace/mmdeploy/tools/deploy.py /root/workspace/mmdeploy/configs/mmdet/panoptic-seg/panoptic-seg_maskformer_tensorrt_static-800x1344.py mask2former_swin-t-p4-w7-224_8xb2-lsj-50e_coco.py best_coco_segm_mAP_50_iter_195000.pth img/demo.jpg --work-dir result_static/ --show --device cuda:0 --dump-info

Environment

06/03 08:11:05 - mmengine - INFO - TorchVision: 0.18.0+cu121
06/03 08:11:05 - mmengine - INFO - OpenCV: 4.9.0
06/03 08:11:05 - mmengine - INFO - MMEngine: 0.10.4
06/03 08:11:05 - mmengine - INFO - MMCV: 2.1.0
06/03 08:11:05 - mmengine - INFO - MMCV Compiler: GCC 11.4
06/03 08:11:05 - mmengine - INFO - MMCV CUDA Compiler: 12.3
06/03 08:11:05 - mmengine - INFO - MMDeploy: 1.3.1+bc75c9d
06/03 08:11:05 - mmengine - INFO - 

06/03 08:11:05 - mmengine - INFO - **********Backend information**********
06/03 08:11:05 - mmengine - INFO - tensorrt:    8.6.1
06/03 08:11:05 - mmengine - INFO - tensorrt custom ops: Available
06/03 08:11:05 - mmengine - INFO - ONNXRuntime: None
06/03 08:11:05 - mmengine - INFO - ONNXRuntime-gpu: 1.15.1
06/03 08:11:05 - mmengine - INFO - ONNXRuntime custom ops:  NotAvailable
06/03 08:11:05 - mmengine - INFO - pplnn:   None
06/03 08:11:05 - mmengine - INFO - ncnn:    None
06/03 08:11:05 - mmengine - INFO - snpe:    None
06/03 08:11:05 - mmengine - INFO - openvino:    None
06/03 08:11:05 - mmengine - INFO - torchscript: 2.3.0+cu121
06/03 08:11:05 - mmengine - INFO - torchscript custom ops:  NotAvailable
06/03 08:11:05 - mmengine - INFO - rknn-toolkit:    None
06/03 08:11:05 - mmengine - INFO - rknn-toolkit2:   None
06/03 08:11:05 - mmengine - INFO - ascend:  None
06/03 08:11:05 - mmengine - INFO - coreml:  None
06/03 08:11:05 - mmengine - INFO - tvm: None
06/03 08:11:05 - mmengine - INFO - vacc:    None
06/03 08:11:05 - mmengine - INFO - 

06/03 08:11:05 - mmengine - INFO - **********Codebase information**********
06/03 08:11:05 - mmengine - INFO - mmdet:   3.2.0
06/03 08:11:05 - mmengine - INFO - mmseg:   None
06/03 08:11:05 - mmengine - INFO - mmpretrain:  None
06/03 08:11:05 - mmengine - INFO - mmocr:   None
06/03 08:11:05 - mmengine - INFO - mmagic:  None
06/03 08:11:05 - mmengine - INFO - mmdet3d: None
06/03 08:11:05 - mmengine - INFO - mmpose:  None
06/03 08:11:05 - mmengine - INFO - mmrotate:    None
06/03 08:11:05 - mmengine - INFO - mmaction:    None
06/03 08:11:05 - mmengine - INFO - mmrazor: None
06/03 08:11:05 - mmengine - INFO - mmyolo:  None

Error traceback

No response

[lansfair]

Do you know how to resolve this issue now? @moliwei2019

[moliwei2019]

Do you know how to resolve this issue now? @moliwei2019 这个地方，mmdeploy把原始应该有的头截断了，你参照mask2former模型在这里把头加上，对应的输入输出改一下，就好了

[lansfair]

请问具体应该如何修改呢，我尝试加上了原有的头，可以成功导出，但得到的输出不对

[moliwei2019]

你想要什么样的输出，加上原来的头之后，就会进行后处理。如果想要scores，labels，masks这些输出的话，直接去这个头的返回结果里取就好了，然后return。转模型的时候再把config那里的输出改成对应的