after using the BertOnnxModelTF.py to optimize the onnx bert model, only support batch_size==1?

onnx / tutorials

Tutorials for creating and using ONNX models

Apache License 2.0

3.37k stars 629 forks source link

after using the BertOnnxModelTF.py to optimize the onnx bert model, only support batch_size==1? #178

Open rigorosyang opened 4 years ago

rigorosyang commented 4 years ago

I use the BertOnnxModelTF to optimize the onnx bert model ,but running the inference occur below exception using batch_size > 1.

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running SkipLayerNormalization node. Name:'SkipLayerNorm1' Status Message: skip is expected to have same shape as input

anyone can help me? Thanks in advance!

tianleiwu commented 4 years ago

@rigorosyang, have you resolved the problem?

I added a Jupyter notebook about Tensorflow Bert model optimization: Inference TensorFlow Bert Model with ONNX Runtime on CPU

vgrabovets commented 4 years ago

I'm facing similar issue. Described it here: https://github.com/microsoft/onnxruntime/issues/4980 There seems to be problem when you use 2 inputs (input_ids, attention_mask) instead of 3 (input_ids, attention_mask, token_type_ids).