search

Tianxiaomo / pytorch-YOLOv4

PyTorch ,ONNX and TensorRT implementation of YOLOv4
Apache License 2.0
4.47k stars 1.49k forks source link

TensorRT(.engine) Conversion Error #151

Open KoPub opened 4 years ago

KoPub commented 4 years ago

Thank you very much for your work.

  1. I obtained ONNX file using 'demo_pytorch2onnx.py'.

  2. ONNX converted into TensorRT(.engine) file through DeepStream 5.0-devel.

  3. The TensorRT(.engine) file was working fine on batch 1~4 however, not on batch 16.

I used Tesla T4.

Error Message is

ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:31 [TRT]: (Unnamed Layer* 419) [Concatenation]: dimensions not compatible for concatenation

ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:31 [TRT]: shapeMachine.cpp (252) - Shape Error in operator(): broadcast iwth incompatible dimensions

ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:31 [TRT]: Instruction: CHECK_BROADCAST 16 2

ERROR: nvdsinfer_backend.cpp:459 Failed to enqueue trt inference batch

ERROR: nvdsinfer_context_impl.cpp:1408 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR

0:48:14.430040294 1955 0x55a5c1a23ad0 WARN nvinfer gstnvinfer.cpp:1188:gst_nvinfer_input_queue_loop: error: Failed to queue input batch for inferencing

ERROR from primary_gie: Failed to queue input batch for inferencing

Additionally, there is a problem that only '.engine' files of the same batch size as the 'ONNX' file are created. (ex. b1.onnx → b1_fp16.engine(O) // b4_fp16.engine(X) b4.onnx → b4._fp16.engine(O) // b1_fp16.engine(X))

p98-ai commented 4 years ago

Hey @KoPub Which command did you use to : "ONNX converted into TensorRT(.engine) file through DeepStream 5.0-devel"

KoPub commented 4 years ago

Hey @KoPub Which command did you use to : "ONNX converted into TensorRT(.engine) file through DeepStream 5.0-devel"

I used [property] option. prop

ersheng-ai commented 4 years ago

@KoPub

Additionally, there is a problem that only '.engine' files of the same batch size as the 'ONNX' file are created. (ex. b1.onnx → b1_fp16.engine(O) // b4_fp16.engine(X) b4.onnx → b4._fp16.engine(O) // b1_fp16.engine(X))

This is not a problem but how it exactly works. Engine directly derives batch size from ONNX. Just simply re-generate ONNX with batch size you expect.

BTW, I will have a look at DeepStream when YoloV4's batch size is larger than 4

KoPub commented 4 years ago

Hello. @ersheng-ai Did you check about engine file larger than 4 in Deepstream 5.0?