Open Pochingto opened 2 years ago
Sorry. As much as I wanted to look into this, I found it difficult for myself to find time to investigate it...
The main reason why I kept MAX_BATCH_SIZE
as a separate constant in "yolo_to_onnx.py" code is that I could set it to -1
(let the batch dimension be dynamic in the ONNX model). Could you help to try if "setting ONNX to batch size -1 and TensorRT to batch size 3" would work for you?
Thx for looking into this. Unfortunately, the batch size is still squared (3x3=9). I would stick to using a batch size of 4 (2x2) for a while, plz take your time on this issue :-)
Hi, thanks for having this awesome repo, it helps a lot when dealing with tensorrt.
I am trying to export a tensorrt model of yolov4-tiny (nc=1) with a batch size of 3, I see there is a
MAX_BATCH_SIZE
parameters in yolo_to_onnx.py and onnx_to_tensorrt.py respectively. Same as #373, if the 2MAX_BATCH_SIZE
parameters are different, an error was thrown.However, with both
MAX_BATCH_SIZE
set to 3, although the model can be exported without error, it actually exports a model with batch size squared. Checking the original model output shape:(8505, )
The exported model output shape:(76545, )
, so the batch size is actually 9 instead of 3.What setting should I use in order to get a correct batch size of 3?
I am providing the error logs below when the two
MAX_BATCH_SIZE
are different: Setting onnx to batch size 3 and tensorrt to batch size 1Setting onnx to batch size 1 and tensorrt to batch size 3