search

open-mmlab / mmdeploy

OpenMMLab Model Deployment Framework
https://mmdeploy.readthedocs.io/en/latest/
Apache License 2.0
2.79k stars 637 forks source link

[Feature] Class agnostic NMS for object detector in torch2onnx #2801

Open darrenjkt opened 4 months ago

darrenjkt commented 4 months ago

Motivation

Currently, overlapping detections of a different class are not filtered out by NMS when converting the mmdetection model to onnx. The class_agnostic flag in the mmdetection model config does not seem to work. I would like to be able to apply a class_agnostic NMS to the output of my detector.

Currently, for RTMDet I have set my model.test_cfg to the following but the class_agnostic flag does not seem to be working (according to my visualized predictions).

{'max_per_img': 300, 'min_bbox_size': 0, 'nms': {'iou_threshold': 0.01, 'type': 'nms', 'class_agnostic': True}, 'nms_pre': 30000, 'score_thr': 0.001}

I've done some digging but I'm not sure where the NMS is currently being encoded into the onnx model so I apologize if this is an implemented feature that I've missed.

Related resources

No response

Additional context

No response

pallavitomra commented 1 week ago

Any resolution that you found?

darrenjkt commented 1 week ago

Not particularly for class agnostic nms but we reduced the number of classes to only predict super classes and it helps to make detections less noisy. I'm thinking a DETR type detector which has no NMS might be better but I have yet to try it.