search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.88k stars 2.14k forks source link

ModelImporter.cpp:750: input: "onnx::Range_464" #2316

Closed Nuwan1654 closed 1 year ago

Nuwan1654 commented 2 years ago

Description

Hi, I am trying to convert yolov5m model into tensort model for batch inference (batch size of 4). but I am getting an error while trexec conversion. This error does not happen when I dont use the --dynamic parameter. I am also not clear that the following steps are correct for generating a tensorRT model for batch inference, but following the documentation, It seems to be the correct way. Any help would be appreciated.

Thanks

Environment

TensorRT Version: 8.4.0.11 NVIDIA GPU: 1024-core NVIDIA Ampere architecture GPU (Jetson Orin) CUDA Version: 11.4 Operating System: Linux ubuntu Python Version (if applicable): 3.8.10 PyTorch Version (if applicable): 1.11.0

files

For exporting yolov5 PyTorch model to ONNX I used https://github.com/ultralytics/yolov5/blob/master/export.py

Steps To Reproduce

  1. generate the yolov5 onnx model and convert the yolov5m.pt to ONNX with --dynamic argument python3 export.py --weights yolov5m_shear.pt --include onnx --data coco128.yaml --dynamic

  2. Convert to tensorRT model with trtexec trtexec --onnx=yolov5m.onnx --fp16 --minShapes=image:1x3x640x640 --optShapes=image:4x3x640x640 --maxShapes=image:8x3x640x640 --saveEngine=yolov5m_dynamic.engine

When converting to tensoRT I get the following error.

[09/09/2022-01:05:31] [E] [TRT] ModelImporter.cpp:748: While parsing node number 290 [Range -> "onnx::Reshape_466"]: [09/09/2022-01:05:31] [E] [TRT] ModelImporter.cpp:749: --- Begin node --- [09/09/2022-01:05:31] [E] [TRT] ModelImporter.cpp:750: input: "onnx::Range_464" input: "onnx::Range_463" input: "onnx::Range_465" output: "onnx::Reshape_466" name: "Range_290" op_type: "Range"

[09/09/2022-01:05:31] [E] [TRT] ModelImporter.cpp:751: --- End node --- [09/09/2022-01:05:31] [E] [TRT] ModelImporter.cpp:753: ERROR: ModelImporter.cpp:162 In function parseGraph: [6] Invalid Node - Range_290 All inputs to range should be initializers. [09/09/2022-01:05:31] [E] Failed to parse onnx file [09/09/2022-01:05:31] [I] Finish parsing network model [09/09/2022-01:05:31] [E] Parsing model failed [09/09/2022-01:05:31] [E] Failed to create engine from model or file. [09/09/2022-01:05:31] [E] Engine set up failed

zerollzeng commented 2 years ago

You can try constant folding the ONNX model: https://github.com/NVIDIA/TensorRT/tree/main/tools/Polygraphy/examples/cli/surgeon/02_folding_constants

Nuwan1654 commented 2 years ago

Thank you for the reply, but even after the constant folding, I am getting the same error.

zerollzeng commented 2 years ago

The reason why is failed it's simple: TRT expect your Range_290's range is an initializer, not a tensor. you can compare your static model and the dynamic model to see if you can make the range shape static and modify it as a constant. e.g. with onnx-graphsurgeon.

ttyio commented 1 year ago

closing since no activity for more than 3 weeks, please reopen if you still have question, thanks!