Open liuxianyi opened 1 year ago
Maybe your gpu hardware is worse than your cpu hardward. Your can have a try with TensorRT.
Optimized models cannot switch hardware, because they are optimized for the specific environment. Read this before using optimized models
Describe the issue
I encountered this warning and I feel that this warning caused my onnx reasoning to work poorly on the GPU.
To reproduce
compare the inference time. i found the performance for onnxruntime-gpu is bad. I think the warning "Graph Optimization level greater than ORT_ENABLE_EXTENDED" cause the bad performance. so, how can l solve this?
Urgency
No response
Platform
Linux
OS Version
Ubuntu 22.04
ONNX Runtime Installation
Released Package
ONNX Runtime Version or Commit ID
onnxruntime-gpu ==1.12.0
ONNX Runtime API
Python
Architecture
X64
Execution Provider
CUDA
Execution Provider Library Version
CUDA 11.1 Cudnn 8.0.5