Open JackWeiw opened 2 months ago
Did you try to see if it works with onnxruntime==1.18?
Did you try to see if it works with onnxruntime==1.18?
I switch to onnxruntime==1.18, there it still return the same error when i try to pre-process
if i simply use quantize_dynamic, it works fine, but it fails to check_model
I set op_version as default(14) when export from PyTorch, my torch version is torch2.3-cu11.8.
Do you have any insights?
Are you using the latest onnx package?
Are you using the latest onnx package?
I have updated onnx to 1.16.1, onnxruntime to 1.18.0, than it succeed in quantization
howerver, when i tried to run it in onnxruntime, it report
This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.
Describe the issue
use shape_inference.quant_pre_process to preprocess will result in error even if i set skip_optimization=True![image](https://github.com/microsoft/onnxruntime/assets/126441921/57c3d34a-144c-4d5f-847b-d9b416f020de)
after that, i use quantize_dynamic, it successfully quantize the model to int8, but it fails to load it back![image](https://github.com/microsoft/onnxruntime/assets/126441921/9006934f-6768-4262-8eeb-f9ae000ff818)
To reproduce
Urgency
Urgent, paper diliver deadline is coming !
Platform
Linux
OS Version
ubuntu22.04
ONNX Runtime Installation
Released Package
ONNX Runtime Version or Commit ID
1.17
ONNX Runtime API
Python
Architecture
X64
Execution Provider
CUDA
Execution Provider Library Version
CUDA11.8