This commit adds the support for a new test suite for special models where we can use custom flags, tolerances, and configurations based on the backend/model compared to iree_tests where we stick with the most basic testing configurations.
In this setup, we don't have to deal with any config files for iree flags.
I also generalized the tools being used in this new test suite, and changed the old test suite to use these generalized tools.
This way we have uniform testing (can expect same behavior from all suites). To do this I created a python package for iree_special_models which is also now pip installed in the workflow file.
The conftest for iree_tests is simple again and has no weird environment variable expansions
Each model test usually has 3 files (scheduled unet has an extra mlir to test the scheduling (don't need those vmfbs anymore)):
1) source mlir
2) json file for what remote files to download
3) python testing file
More info can be found in the README.md for this directory
This commit adds the support for a new test suite for special models where we can use custom flags, tolerances, and configurations based on the backend/model compared to iree_tests where we stick with the most basic testing configurations. In this setup, we don't have to deal with any config files for iree flags. I also generalized the tools being used in this new test suite, and changed the old test suite to use these generalized tools. This way we have uniform testing (can expect same behavior from all suites). To do this I created a python package for iree_special_models which is also now pip installed in the workflow file. The conftest for iree_tests is simple again and has no weird environment variable expansions
Each model test usually has 3 files (scheduled unet has an extra mlir to test the scheduling (don't need those vmfbs anymore)): 1) source mlir 2) json file for what remote files to download 3) python testing file
More info can be found in the README.md for this directory