[BERT-Squad][opset-8][Model Zoo] Inference fails on bert/embeddings/one_hot op

[mbencer]

Describe the bug I've tried to run BERT-Squad (opset8) from model zoo (https://github.com/onnx/models/blob/master/text/machine_comprehension/bert-squad/model/bertsquad-8.tar.gz). Unfortunately, I see "INVALID_ARGUMENT : Non-zero status code returned while running Gather node. Name:'bert/embeddings/one_hot' Status Message: indices element out of data bounds, idx=101 must be within the inclusive range [-2,1]" exception during inference.

System information

• Linux Ubuntu Ubuntu 16.04.7 LTS
• onnx 1.4.1
• onnxruntime 1.6.0 (from pip)

To Reproduce

import onnxruntime as rt
import numpy as np
import onnx
import onnx.numpy_helper

base_path = "/home/mbencer/workspace/test_bert/download_sample_8/"
model_path = base_path + "bertsquad8.onnx"

sess = rt.InferenceSession(model_path)

input_names = [sess.get_inputs()[i].name for i in range(len(sess.get_inputs()))]
output_names = [sess.get_outputs()[i].name for i in range(len(sess.get_outputs()))]

input_data = []
for i in range(len(sess.get_inputs())):
    proto_bytes = open(base_path+"test_data_set_0/input_{0}.pb".format(i), 'rb').read()
    tensor_proto = onnx.TensorProto.FromString(proto_bytes)
    data_array = onnx.numpy_helper.to_array(tensor_proto)
    input_data.append(data_array)

inputs_map = dict(zip(input_names, input_data))
pred = sess.run(output_names, inputs_map)

for idx, val in enumerate(output_names):
    print(val)

Printed exception

2021-02-03 13:14:57.953582320 [E:onnxruntime:, sequential_executor.cc:333 Execute] Non-zero status code returned while running Gather node. Name:'bert/embeddings/one_hot' Status Message: indices element out of data bounds, idx=101 must be within the inclusive range [-2,1]
Traceback (most recent call last):
  File "bert8_ort.py", line 22, in <module>
    pred = sess.run(output_names, inputs_map)
  File "/home/mbencer/.virtualenvs/ov/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 124, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Gather node. Name:'bert/embeddings/one_hot' Status Message: indices element out of data bounds, idx=101 must be within the inclusive range [-2,1]

If I try to run model using ./onnx_test_runner it also fails.

[faxu]

does the opset10 version from the zoo work? any reason you need 8?

[mbencer]

I think that expected data from model zoo are incorrect for berstsquad-8 (I run it using other frameworks). It is topic connected with https://github.com/microsoft/onnxruntime/issues/6429. I want to update this expected data in model zoo but as I understand I should use onnxruntime for it.

[GeorgeS2019]

@faxu @mbencer I am attempting to summarize the issues of using bertsquad-10 test input and output data to help evaluate the bertsquad-10 inference

I have not checked the bertsquad-8 version yet. I would recommend focusing on bertsquad-10 to resolve outstanding issues.

[GeorgeS2019]

@faxu it is unclear if I have issue with using .NET to inference Bert.onnx, should I post here under OnnxRuntime or OnnxModel which provide the Bert.onnx model.

[stale[bot]]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.