search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.15k stars 2.08k forks source link

Detectron2 - no detections below 832x832 in trt, onnxruntime versions detect correctly #3932

Open Huxwell opened 3 weeks ago

Huxwell commented 3 weeks ago

Hey!

I've been trying square resolutions divisible by 64x64 (divisibility by 32 was not always enough for error-free conversion). I'm doing this both for maskrcnn with coco weights, as well as custom keypoints with similar outcome. The issue is, 1344x1344, 1280x1280, 960x960, 832x832 don't seem to loose accuracy, but 768x768, 704x704, 640x640 are fully blind (no detections or detections with confidence from activation function below 10%). In onnxruntime the 640x640, 768x768 work perfectly too, so the problem is not in detectron2 export_model.py.

This leads to conclusion, that something in create_onnx.py or in one of the trt plugins (ROIAlign or Efficient NMS) uses width and height of the image as basis for some computation that gives different results 832x832. Any tips on how to look for the solution @azhurkevich @letmejoing @RajUpadhyay ?

Related issues: https://github.com/NVIDIA/TensorRT/issues/2142 https://github.com/NVIDIA/TensorRT/issues/3792#issuecomment-2135726159 https://github.com/facebookresearch/detectron2/issues/5255

RajUpadhyay commented 3 weeks ago

@Huxwell I have actually not touched detectron2 for quite a while but from what I remember, isn't detectron2's default configuration settings for min size = 800 & max size = 1300?

While I do understand that onnxruntime works for your onnx file generated from export_model.py but how are you running your tensorrt? Is it deepstream? If no then, Is it the infer.py given to us by this repo? If yes then are you modifying the preprocessing where the model again tries to get the detectron2's default min_size (800) & max_size (1300).

Since the information is not enough, it is really difficult to narrow down your situation but from what I see, your models which have the dimensions less than 800 are not being supported.

Find a way to validate your onnx file generated from create_onnx.py. i.e., netron or maybe TensorRTEP (onnxruntime) Validate your tensorrt engine using trtexec i.e., loadEngine.

I could be wrong but maybe you could try checking all this if you haven't already. Thanks.

Huxwell commented 3 weeks ago

Thanks for the tips.

I am running the model with infer.py If I add print statement to Line 129 with scores: https://github.com/NVIDIA/TensorRT/blob/release/9.0/samples/python/detectron2/infer.py#L127 I will get 100 scores, sorted. For resolution >= 832, there will be some high values:

[[0.9966532  0.4419432  0.43740156 0.39249143 0.24443977 0.16181383
  0.15681778 0.11954353 0.11906159 0.07982704 0.06165875 0.06028476
  0.05638076 0.05417441 0.05292124 0.         0.         0. .......................

But for <832 (i.e. 768x768) it's:

[[0.00332  0.000143  0.0 0.0 0.0 0.0
  0.0 0.0 0.0 0.0 0.0 0.0
  0.0 0.0 0.0 0.         0.         0.

So I assume it's not a postprocesing issue and infer.py is correct.

TensorRTEP (onnxruntime) Do you think it will work without ROIAlign and NMS as tensor plugins from create_onnx.py? I will check, as well as try to validate with validators.

I also thought about 800 and 1300 being default in detectron2, but I had no problems with 1344x1344 in tensorrt, and no problems with 640x640 in onnx_runtime, so the problem is most likely in tensorrt conversion.