triton server unable to create TensorRT engine

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html

BSD 3-Clause "New" or "Revised" License

8.07k stars 1.45k forks source link

triton server unable to create TensorRT engine #7028

Open wfd2022 opened 6 months ago

wfd2022 commented 6 months ago

Description When I followed the official guidance to convert the ONNX model to TensorRT format and started the Triton Server, I encountered the following error

Triton Information What version of Triton are you using? tritonserver:21.11-py3 tensorrt:21.11-py3 Are you using the Triton container or did you build it yourself? Triton container To Reproduce

rmccorm4 commented 5 months ago

Hi @wfd2022,

Can you share the steps you took to build your engine?
21.11 is 2.5 years old at this point so it will be hard to support. Can you please try a recent version like 24.01 or 24.02 to see if this is still reproducible?