Set up a regression test suite

[ScottTodd]

We can think of "regression tests" as a middle layer between unit tests and integration tests:

• unit tests: hand-written, small, fast to run
• regression tests: reduced from full models but still complex, medium size, medium run time
• integration tests: full models, large, slow to run

When an issue is detected and fixed, we should have a process to:

1. Minimize and reproduce the issue (using the tools below, or some other process)
2. Add the [minimal] reproducer to the regression suite
3. Run the regression suite regularly (every commit, nightly, etc.)

Tools like fuzzers can also be used to generate regression test cases.

Tools to help

• MLIR "reproducers": https://mlir.llvm.org/docs/PassManagement/#crash-and-failure-reproduction to collect IR snippets, pipelines, and flags from failed compiler runs
• iree-reduce: https://github.com/iree-org/iree/pull/14892
• mlir-reduce: https://mlir.llvm.org/docs/Tools/mlir-reduce/

Implementation ideas

• pytest? CMake / CTest?
• See also the existing regression tests in https://github.com/iree-org/iree/tree/main/tests/e2e/regression

[ScottTodd]

Textual MLIR using most of our dialects (e.g. linalg) is not generally stable, so this sort of test suite does run the risk of breaking regularly. If we add enough test cases within each window when the op semantics / serialized representations do not change then this could still be useful enough.