Enable OnnxRuntime to use execution providers for benchmarking.

[ramkrishna2910]

ONNX Runtime works with different hardware acceleration libraries through its extensible Execution Providers (EP) framework to optimally execute the ONNX models on the hardware platform. Currently, mlagility only supports the default execution provider (CPU/ MLAS). Ability to use the other EPs can broaden the spectrum for benchmarking.

[ramkrishna2910]

The following is the list of all available onnxruntime execution providers

The focus of this MR is to add support for:

• NVIDIA CUDA
• AMD ROCm
• Intel OpenVino

cc @jeremyfowers

[jeremyfowers]

@ramkrishna2910 how hard is it to add one of the ARM approaches in there?

[ramkrishna2910]

None of the ARM EPs listed here has a pypi package ready to deploy. We will have to build onnxruntime from source within the container we plan to deploy. The downside is the initial setup time for building from source will be ~15-20 mins.

[jeremyfowers]

Let's say we took that install-from-source as a manual step. And provisioning an ARM server on AWS or Azure as a manual step. Any difficulty beyond that?

[ramkrishna2910]

Nope. Provisioning and install from source are the long poles. The other parts of the pipeline are pretty straightforward.