search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
13.62k stars 2.78k forks source link

[Performance] Is my script set to get optimal performance of onnxruntime? #20945

Open JackWeiw opened 1 month ago

JackWeiw commented 1 month ago

Describe the issue

I am profiling performance of onnx model converted from PyTorch2.3.0-cu11.8, it shows that the performance is little bit slower than the torch version. Is there something i missed?

To reproduce

script image image

Urgency

Really urgent, paper deadline is incoming!!!

Platform

Linux

OS Version

ubuntu22.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.18.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

CUDA11.8, onnx1.16.1

Model File

No response

Is this a quantized model?

No

github-actions[bot] commented 1 week ago

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.