run error in benchmark_pipelines.ipynb on Win 10

[baiziyuandyufei]

when I run onnx_transformers\notebooks\benchmark_pipelines.ipynb，I got error below: onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type. only use nlp_torch = pipeline("feature-extraction", onnx=False)，get the inference time figure.

[patil-suraj]

hi @baiziyuandyufei did you get this error after running nlp_torch = pipeline("feature-extraction", onnx=False) ?

[baiziyuandyufei]

thanks @patil-suraj if only run nlp_torch = pipeline("feature-extraction", onnx=False), no problem. but run nlp_onnx = pipeline("feature-extraction", onnx=True), will get error.

[patil-suraj]

Could you post your env info (transformers version, onnruntime version, torch version etc) ? I ran the notebook again and couldn't reproduce the issue.

[baiziyuandyufei]

My operating system environment is Win10 Professional. My env info: >pip list Package Version ----------------- --------- certifi 2020.6.20 chardet 3.0.4 click 7.1.2 coloredlogs 14.0 cycler 0.10.0 dataclasses 0.7 filelock 3.0.12 future 0.18.2 humanfriendly 8.2 idna 2.10 joblib 0.16.0 kiwisolver 1.2.0 matplotlib 3.3.1 numpy 1.19.1 onnx 1.7.0 onnx-transformers 0.1.0 onnxruntime 1.4.0 onnxruntime-tools 1.4.2 packaging 20.4 pandas 1.1.2 Pillow 7.2.0 pip 20.2.3 protobuf 3.13.0 psutil 5.7.2 py-cpuinfo 7.0.0 py3nvml 0.2.6 pyparsing 2.4.7 pyreadline 2.1 python-dateutil 2.8.1 pytz 2020.1 regex 2020.7.14 requests 2.24.0 sacremoses 0.0.43 scipy 1.5.2 seaborn 0.11.0 sentencepiece 0.1.91 setuptools 36.4.0 six 1.15.0 tokenizers 0.8.1rc2 torch 1.6.0+cpu torchvision 0.7.0+cpu tqdm 4.48.2 transformers 3.1.0 typing-extensions 3.7.4.3 urllib3 1.25.10 wheel 0.29.0 wincertstore 0.2 xmltodict 0.12.0

[baiziyuandyufei]

@patil-suraj I don't know if I'm right. I estimate the reason is that the distilbert-base-cased model has not been download. Under MacOS, It will automatically download the distilbert-base-cased file, (I dowload the model manually, and set the pipline's parameter model='./distilbert-base-cased') run it again, there's no problem. the distilbert-base-cased model file include: config.json modelcard.json pytorch_model.bin tf_model.h5 these file can be downlaoded from https://huggingface.co/distilbert-base-cased#list-files and need download the vocab.txt the vocab.txt can be downloaded from https://huggingface.co/bert-base-cased#list-files

[baiziyuandyufei]

With Win10, errors occur even when the local model is used.

[patil-suraj]

I haven't tested on windows not sure about onnx support on windows, so can't much here

[baiziyuandyufei]

The problem is in the numpy array type, which defaults to INT32 in Windows, modify the function _forward_onnx in onnx_transformers\pipelines.py file. modified detail:

   def _forward_onnx(self, inputs, return_tensors=False):
        # inputs_onnx = {k: v.cpu().detach().numpy() for k, v in inputs.items() if k in self.input_names}
        # inputs_onnx = {k: v for k, v in inputs.items() if k in self.input_names}
        inputs_onnx = {k: v.astype(np.int64) for k, v in inputs.items() if k in self.input_names}
        predictions = self.onnx_model.run(None, inputs_onnx)
        return predictions

[patil-suraj]

Thank you very much @baiziyuandyufei ! I'll run tests again with this change and update the code.