SOFIE is designed for fast inference, that's why having a reference benchmark with the state-of-the-art ONNX processing framework (ONNX Runtime) comes in handy for asserting the performance of the models emitted by SOFIE.
Integrating ONNX Runtime
The official repository still doesn't provide a CMake script for automatically finding ORT in the system, that's why I wrote a basic package finder.
[x] CMake package finder (a.k.a. FindONNXRuntime.cmake).
[x] GBenchmark for ONNX models with ONNXRuntime libraries.
[x] Manual GBenchmark for models emitted by SOFIE, Linear models only.
[x] Automating the compilation of models with SOFIE.
Notes
For every ONNX model in input_models a gbenchmark is automatically created.
For every .hxx file in input_models/compiled a gbenchmark for SOFIE is automatically created.
All of this happens by means of BENCHMARK_CAPTUREs configured a CMake script.
Benchmarks of models with ONNX Runtime and SOFIE
SOFIE is designed for fast inference, that's why having a reference benchmark with the state-of-the-art ONNX processing framework (ONNX Runtime) comes in handy for asserting the performance of the models emitted by SOFIE.
Integrating ONNX Runtime
The official repository still doesn't provide a CMake script for automatically finding ORT in the system, that's why I wrote a basic package finder.
FindONNXRuntime.cmake
).Notes
For every ONNX model in
input_models
a gbenchmark is automatically created.For every
.hxx
file ininput_models/compiled
a gbenchmark for SOFIE is automatically created. All of this happens by means ofBENCHMARK_CAPTURE
s configured a CMake script.