search

nbigaouette / onnxruntime-rs

Rust wrapper for Microsoft's ONNX Runtime (version 1.8)
Apache License 2.0
276 stars 99 forks source link

Bindings float precision #81

Open PPakalns opened 3 years ago

PPakalns commented 3 years ago

It looks like somewhere these bindings are adding error to float output.

When using native C++ onnxruntime API in the output for yolov3 onnx model I receive

Using python and native C++ onnxruntime API:

[[[     4.5295      4.3708      8.5376       11.65  2.2829e-05     0.98627]
  [     11.613      3.9123      22.815      9.5377  2.1607e-05     0.98834]
  [     19.874      4.0934      34.348      8.3545  1.0371e-05     0.98514]
  ...

Using onnxruntime-rs API:

[[[4.4851866, 4.155593, 8.946828, 12.582257, 0.0000130140525, 0.986418],
  [11.476209, 3.759369, 23.092682, 9.8777075, 0.000009529847, 0.9887239],
  [19.693684, 3.971115, 34.107414, 8.499779, 0.0000042610786, 0.9855219],
...

These results are approximately the same, but you can see the errors, which are pretty huge!

I even tried to compile onnxruntime with different backends and enable them with modifying this crate.

Tried download and system onnxruntimes, with and without GPU, with custom backend, etc.

haixuanTao commented 3 years ago

Hey, just for info, I've runned a custom Bert Model on onnxruntime using HF tokenizer on Python and Rust and the result are identical. I've runned it on a personal onnxruntime v1.8 with GPU which could be the reason of the fix.

My code if it can help: https://github.com/haixuanTao/bert-onnx-rs-pipeline