search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.79k stars 2.94k forks source link

Performance comparison #5834

Open meedeepak opened 4 years ago

meedeepak commented 4 years ago

Describe the bug I ran the official tutorial code for onnx

[(https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/notebooks/PyTorch_Bert-Squad_OnnxRuntime_CPU.ipynb)]

Didn't change any of the code and ran it as it is

My observations were: PyTorch cpu Inference time = 132.92 ms OnnxRuntime cpu Inference time = 115.03 ms

The difference of 15 ms does not seem pretty impressive as compared to the efforts needed to migrate to onnx. Please suggest something or am I missing some other configuration.

System information

To Reproduce Run [(https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/notebooks/PyTorch_Bert-Squad_OnnxRuntime_CPU.ipynb)]

Expected behavior 17x faster performance image

zhanghuanrong commented 4 years ago

Thanks for reporting issues. Keep improving the performance is our goal. We will investigate what you described.