deBERTaV3 Error with [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:token_type_ids

[serdarildercaglar]

System Info

onnx runtime version: 1.13.1

Who can help?

@JingyaHuang @echarlaix

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [X] My own task or dataset (give details below)

Reproduction

Hello, Could you pls check the below error? I tried this a lot of times in this pipeline with different deberta models.

hg_checkpoint = "microsoft/deberta-v3-base" save_hg = "tmp/hg_onnx/"

Load a model from transformers and export it to ONNX ort_model_hg = ORTModelForTokenClassification.from_pretrained(hg_checkpoint, from_transformers=True) tokenizer_hg = AutoTokenizer.from_pretrained(hg_checkpoint)

Save the onnx model and tokenizer ort_model_hg.save_pretrained(save_hg) tokenizer_hg.save_pretrained(save_hg)

Define the quantization methodology qconfig = AutoQuantizationConfig.arm64(is_static=False, per_channel=False) quantizer_hg = ORTQuantizer.from_pretrained(ort_model_hg)

Apply dynamic quantization on the model

quantizer_hg.quantize(save_dir=save_hg, quantization_config=qconfig) from optimum.onnxruntime import ORTModelForTokenClassification from transformers import pipeline, AutoTokenizer

model_hg = ORTModelForTokenClassification.from_pretrained(save_hg, file_name="model_quantized.onnx") tokenizer_hg = AutoTokenizer.from_pretrained(save_hg) pipeline_hg = pipeline("token-classification", model=model_hg, tokenizer=tokenizer_hg, aggregation_strategy = 'first') results = pipeline_hg(text) results

InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:token_type_ids

Expected behavior

I tried this pipeline with BERT and I didn't face any problems.

[michaelbenayoun]

I was able to reproduce the bug, will look into that ASAP!

[michaelroyzen]

Any updates @michaelbenayoun

[michaelroyzen]

I am facing this issue as well

[neibla]

In the interim you can manually fix the tokenizer model input names before creating the pipeline

tokenizer.model_input_names = ["input_ids", "attention_mask"]