search

triton-inference-server / openvino_backend

OpenVINO backend for Triton.
BSD 3-Clause "New" or "Revised" License
30 stars 16 forks source link

ERROR: Failed to create instance: openvino error in retrieving original shapes from output boxes : get_shape was called on a descriptor::Tensor with dynamic shape #84

Open dineshelavazhagan opened 2 weeks ago

dineshelavazhagan commented 2 weeks ago

tried to load openvino public model - person-detection-0303

with config name: "person-detection-0303" backend: "openvino" max_batch_size: 0 input [ { name: "image" data_type: TYPE_FP32 dims: [ 1, 3, 720,1280] } ] output [ { name: "boxes" data_type: TYPE_FP32 dims: [ 100,5] } ]

instance_group [ { count: 1 kind: KIND_CPU } ]

parameters: [ { key: "NUM_STREAMS" value: { string_value: "1" } }, { key: "RESHAPE_IO_LAYERS" value: { string_value:"yes" } }, { key: "SKIP_OV_DYNAMIC_BATCHSIZE" value: { string_value:"yes" } }, { key: "PERFORMANCE_HINT" value: { string_value: "LATENCY" } } ]

dtrawins commented 3 days ago

As of now openvino_backend doesn't support model with a range in the input and output shapes. That is the case for model person-detection-0303. Is has on output the shapes boxes([0~100],5) and labels ([0~100])