Onnx runtime error with input_shape_size == size was false

[gsasikiran]

When i try to load and run the onnx model, I am getting the following error message. I ran the code from https://github.com/urchade/GLiNER/blob/main/examples/convert_to_onnx.ipynb to save as onnx model.

This is how I am loading the model model = GLiNER.from_pretrained("gliner_multi-v2.1", load_onnx_model=True, load_tokenizer=True)

2024-06-28 10:07:16.986464308 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running Reshape node. Name:'/span_rep_layer/span_rep_layer/Reshape_2' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:45 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, onnxruntime::TensorShapeVector&, bool) input_shape_size == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{1,36,512}, requested shape:{1,17,12,512}

Traceback (most recent call last): File "/home/project_path/src/extract_locations.py", line 138, in extract_location entities = model.predict_entities(normalized_phrase.lower(), labels, threshold=0.3) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/project_path/lib/python3.11/site-packages/gliner/model.py", line 176, in predict_entities return self.batch_predict_entities( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/project_path/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/project_path/lib/python3.11/site-packages/gliner/model.py", line 198, in batch_predict_entities model_output = self.model(model_input)[0] ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/project_path/lib/python3.11/site-packages/gliner/onnx/model.py", line 59, in call return self.forward(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/project_path/lib/python3.11/site-packages/gliner/onnx/model.py", line 87, in forward inference_output = self.run_inference(prepared_inputs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/project_path/lib/python3.11/site-packages/gliner/onnx/model.py", line 47, in run_inference onnx_outputs = self.session.run(None, inputs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/anaconda3/envs/query-intent-recognition/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run return self._sess.run(output_names, input_feed, run_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Reshape node. Name:'/span_rep_layer/span_rep_layer/Reshape_2' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:45 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, onnxruntime::TensorShapeVector&, bool) input_shape_size == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{1,36,512}, requested shape:{1,17,12,512}

[Ingvarstep]

@gsasikiran , thank you very much for finding the issue. It looks like ONNX doesn't support some operations of the new features selection function. I already fixed it and the proper fix will be available in the next release.

[wang-haoxian]

@Ingvarstep Thank you for your prompt resolution of the problem! Would you mind telling me when the fix will be released please?

[Ingvarstep]

@wang-haoxian , you can find it in fixes branch, I would appreciate if you can test it on your end

[wang-haoxian]

@wang-haoxian , you can find it in fixes branch, I would appreciate if you can test it on your end

Thank you so much! I will let you know if it works.

[wang-haoxian]

By using the python code with load_onnx_model=True it does run.(edit: it doesn't work, but a different error

InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running ScatterND node. Name:'/ScatterND_1' Status Message: invalid indice found, indice = 10

)

However in my case, I used the notebook in the example to export the model and used it in Triton Inference Server 24.04-py3 I still got the same error as the issue

 [E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running Reshape node. Name:'/span_rep_layer/span_rep_layer/Reshape_2' Status Message: /workspace/onnxruntime/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:44 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, onnxruntime::TensorShapeVector&, bool) input_shape_size == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{1,84,512}, requested shape:{1,10,12,512}

I am using urchade/gliner_multi-v2.1

[Ingvarstep]

@wang-haoxian, did you get this error constantly on each example or on specific input examples?

[wang-haoxian]

@wang-haoxian, did you get this error constantly on each example or on specific input examples?

It's constantly on each example with different labels. I used your notebook for exporting.

onnx_model.batch_predict_entities(["Hello now who is Fast and Furious", "I am in Paris"], labels=["loc", "name", "org"])

I tried to variate the sentences and the labels for testing.

[geraldstanje]

what happens if you convert the pytorch model to onnx yourself? does that work? what happens if you use urchade/gliner_base?

also does gliner_model = GLiNER.from_pretrained("gliner_medium", load_tokenizer=True) have some labels of datatype string - if yes how to change those labels to datatype integer?

cc @Ingvarstep