search

iree-org / iree-test-suites

Test suites for IREE and related projects
Apache License 2.0
2 stars 6 forks source link

Drop matmul "compilation_info" generator code and test cases. #15

Closed ScottTodd closed 2 months ago

ScottTodd commented 2 months ago

Progress on https://github.com/iree-org/iree-test-suites/issues/2

(This is not yet running on CI anywhere, still refactoring the test suite before passing it back into developer hands)

This was leaking compiler implementation details into test cases. The compiler should be selecting a pipeline on its own based on the target device(s). If a developer wants to try a different pipeline, they can make a code change, introduce a dedicated flag (global/hacky), or use a transform dialect script.

I think this drops some of the rocm test suites, which were only added using explicit compilation_info settings. Leaving adding that back to a follow-up / further refactoring. Editing the 2000 line CMakeLists.txt file (generated from a 700 line BUILD.bazel file) is error-prone especially with how this file has grown over the years. If we stick to CMake/CTest, it probably makes sense to start the file over from scratch.

ScottTodd commented 2 months ago

In the PR description, could you clarify if we still tests the same matmul shapes (including data types)?

There's a note there already. This probably drops some coverage. I'm going to continue refactoring the test suite then add coverage back across the full set of backends and test cases in some new format.

ScottTodd commented 2 months ago

Ping @erman-gurses ?