About inference efficiency of ONNX with QNN EQ

microsoft / onnxruntime-inference-examples

Examples for using ONNX Runtime for machine learning inferencing.

MIT License

1.07k stars 312 forks source link

About inference efficiency of ONNX with QNN EQ #402

Open shawnyxf opened 4 months ago

shawnyxf commented 4 months ago

I wonder if anyone have tested the efficiency of ONNX with QNN EQ on cpu or htp?

Or how much faster for inference using the ONNX with QNN EQ than just ONNX alone?

HectorSVC commented 3 months ago

It's so high level. It depends on which Qualcomm hardware you are using. You can add some code around the session run to get the number for your device.