How to inference with batch size > 1?

[anidh]

Hi There,

I need to infer with a batch size of 2. I have exported the model to onnx format using the command -

yolo export model=best.pt format=onnx simplify=True opset=11 dynamic=True

This generates an onnx file with a batch size of -1 . I need help in converting this onnx file to an engine file which can have a batch size of 2 as well as help in inferencing of this engine file. I tried to generate the engine file directly from the ultralytics repo using the command

yolo export model=best.pt format=engine dynamic=True simplify=True opset=11

Then using this engine file with your repo gives the following error -

Traceback (most recent call last): File "/home/anidhsingh/YOLOv8-TensorRT/batch.py", line 87, in <module> main(args) File "/home/anidhsingh/YOLOv8-TensorRT/batch.py", line 44, in main bboxes, scores, labels = det_postprocess(data) File "/home/anidhsingh/YOLOv8-TensorRT/models/torch_utils.py", line 37, in det_postprocess assert len(data) == 4 AssertionError

Can you help infer with a batch size of 2 and how to generate the engine file for that? Thanks again for the great work on this repo.

[1224wxwx]

same question.

[triple-Mu]

same question.

Great I will add batch inference soon.

[AnaRhisT94]

Have you added batch inference? @triple-Mu