dynamic shape onnx error

[chenchunhui97]

Description I have onnx files to deploy using tritonserver:24.01-py3, but encountered error. as the shape on onnx input/output is dynamic, instead of fixed. so maybe the config.pbtxt is incorrect. （I use model_analyzer to analyze the performance of this model, but launch failed）

Triton Information use tritonserver images to deploy directly, of version 24.01-py3

Are you using the Triton container or did you build it yourself? No

To Reproduce

this is input of onnx model:

and output:

my own config.pbtxt:

name: "rec_onnx"
platform: "onnxruntime_onnx"
max_batch_size : 8

input [
  {
    name: "x"
    data_type: TYPE_FP32
    format: FORMAT_NCHW
    dims: [ 3, 48, 320 ]
  }
]
output [
  {
    name: "softmax_11.tmp_0"
    data_type: TYPE_FP32
    dims: [ 40, 6625 ]
  }
]

Expected behavior

can launch the model and call it successfully. (I need to set a specific shape for model inference when profiling this model.)

[chenchunhui97]

error log:

Model rec_performance load failed: [StatusCode.INTERNAL] failed to load 'rec_performance', failed to poll from model repository

[chenchunhui97]

Actually tritonserver can launch the model without config.pbtxt, but I want to analyze the model prerformance so I must specific the input and output shape.

[chenchunhui97]

name: "rec_onnx"
platform: "onnxruntime_onnx"
max_batch_size : 8

input [
  {
    name: "x"
    data_type: TYPE_FP32
    format: FORMAT_NCHW
    dims: [ 3, 48, 320 ]
  }
]
output [
  {
    name: "softmax_11.tmp_0"
    data_type: TYPE_FP32
    dims: [ -1, 6625 ]
  }
]

seems work