iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.57k stars 574 forks source link

Update CI to include benchmarking changes in Test Suite. #17655

Closed saienduri closed 3 months ago

saienduri commented 3 months ago

This commit updates the SHARK-Test ref, config files, and yaml files to have the most up to date flags and benchmarking support. I will also concentrate on a python implementation for pulling in configs in Test-Suite, so we don't have to rely on all these config files. Checked the golden values over 15 times in the CI so should be good.

This commit also adds support to the CI, so that it generates a job summary of the benchmark mean times for e2e and all the sub models. This can be seen by developers in the summary tab of the PckgCI testing. Example: https://github.com/iree-org/iree/actions/runs/9501523985

image

Side note: The build_test_all_bazel was failing the first couple times and then passed. Seems to be unstable