Open kobzaond opened 2 years ago
Thanks for reporting! Could you please share link for the model for further investigation?
Here is the link: https://huggingface.co/bert-base-uncased, note, that the model has a single (classification) linear layer on top
This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.
Any updates for this issue? I met the same one. @zhanghuanrong
Here is environments: ort-nightly-gpu==1.13.0.dev20221005006 onnx==1.9.0
Same model(https://huggingface.co/bert-base-uncased) when I run preprocess.:
python -m onnxruntime.quantization.preprocess --input bert.opt.onnx --output bert.opt.pre_process.onnx
Same issue, none of my models can run successfully by onnxruntime.quantization.preprocess
, but they work by onnxruntime inference session.
same issue ! Cannot convert my onnx model to tensorrt, so I tried to use the symbolic_shape_infer.py script but it raises the Exception: Incomplete symbolic shape inference
Same issue. And not only for transformer model.
Same issue with transformers and diffusional models
same issue with codeformer upscaler
I solved this problem by running optimization first and then quantization, hope this helps
Fisrt runpython -m onnxruntime.transformers.optimizer --input=release/class.onnx --output=release/class.ops.onnx
Then runpython -m onnxruntime.quantization.preprocess --input release/class.ops.onnx --output release/class.ops-infer.onnx
onnx==1.14.1
I solved this problem by running optimization first and then quantization, hope this helps Fisrt run
python -m onnxruntime.transformers.optimizer --input=release/class.onnx --output=release/class.ops.onnx
Then runpython -m onnxruntime.quantization.preprocess --input release/class.ops.onnx --output release/class.ops-infer.onnx
onnx==1.14.1
This does not work for me on onnx==1.14.1 and version 1.16.1
I solved this problem by running optimization first and then quantization, hope this helps Fisrt run
python -m onnxruntime.transformers.optimizer --input=release/class.onnx --output=release/class.ops.onnx
Then runpython -m onnxruntime.quantization.preprocess --input release/class.ops.onnx --output release/class.ops-infer.onnx
onnx==1.14.1
I tried this out for a custom coded pytorch transformer model. The first command runs, but the second does not. It throws the same error Exception: Incomplete symbolic shape inference
.
My onnx versions:
onnx 1.16.2
onnxruntime 1.19.2
onnxscript 0.1.0.dev20240918
When I convert BERT (pytorch model) to onnx format (without any optimizations) and then try to run the "symbolic_shape_infer.py" script with the obtained onnx model as an input argument, I get the following error: File "symbolic_shape_infer.py", line 2096, in
args.guess_output_rank, args.verbose)
File "symbolic_shape_infer.py", line 2062, in infer_shapes
raise Exception("Incomplete symbolic shape inference")
Exception: Incomplete symbolic shape inference
The Bert model is with one classification layer on top (2 classes, random initialization, no fine-tuning). So literally BertForSequenceClassification.from_pretrained('bert-base-uncased'). Note, that inference with the onnx model through onnxruntime works.