Open KoPub opened 4 years ago
Hey @KoPub Which command did you use to : "ONNX converted into TensorRT(.engine) file through DeepStream 5.0-devel"
Hey @KoPub Which command did you use to : "ONNX converted into TensorRT(.engine) file through DeepStream 5.0-devel"
I used [property] option.
@KoPub
Additionally, there is a problem that only '.engine' files of the same batch size as the 'ONNX' file are created. (ex. b1.onnx → b1_fp16.engine(O) // b4_fp16.engine(X) b4.onnx → b4._fp16.engine(O) // b1_fp16.engine(X))
This is not a problem but how it exactly works. Engine directly derives batch size from ONNX. Just simply re-generate ONNX with batch size you expect.
BTW, I will have a look at DeepStream when YoloV4's batch size is larger than 4
Hello. @ersheng-ai Did you check about engine file larger than 4 in Deepstream 5.0?
Thank you very much for your work.
I obtained ONNX file using 'demo_pytorch2onnx.py'.
ONNX converted into TensorRT(.engine) file through DeepStream 5.0-devel.
The TensorRT(.engine) file was working fine on batch 1~4 however, not on batch 16.
I used Tesla T4.
Error Message is
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:31 [TRT]: (Unnamed Layer* 419) [Concatenation]: dimensions not compatible for concatenation
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:31 [TRT]: shapeMachine.cpp (252) - Shape Error in operator(): broadcast iwth incompatible dimensions
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:31 [TRT]: Instruction: CHECK_BROADCAST 16 2
ERROR: nvdsinfer_backend.cpp:459 Failed to enqueue trt inference batch
ERROR: nvdsinfer_context_impl.cpp:1408 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR
0:48:14.430040294 1955 0x55a5c1a23ad0 WARN nvinfer gstnvinfer.cpp:1188:gst_nvinfer_input_queue_loop: error: Failed to queue input batch for inferencing
ERROR from primary_gie: Failed to queue input batch for inferencing
Additionally, there is a problem that only '.engine' files of the same batch size as the 'ONNX' file are created. (ex. b1.onnx → b1_fp16.engine(O) // b4_fp16.engine(X) b4.onnx → b4._fp16.engine(O) // b1_fp16.engine(X))