This commit refactors the code here to own "main()". It adds a main()
very similar to the one in torch-mlir's
e2e_testing/torchscript/main.py, but they are expected to diverge over
time (e.g. configuring different HAL targets and such, configuring
devices for running tests on devices, ec.), so there wasn't much point
in sharing code.
In this initial rewrite, support for dylib and vmvx backends are
added. CUDA and Vulkan are TODO.
The "external config" functionality didn't provide enough flexibility --
putting appropriate framework functionality in a library and letting the
downstream "own main()" works much better.
There seem to be about 10-ish issues (not all of them in IREE -- some
are improvements to our IREE test config to properly handle multiple
results/etc.).
This commit refactors the code here to own "main()". It adds a main() very similar to the one in torch-mlir's e2e_
testing/torchscript/main.py, but they are expected to diverge over time (e.g. configuring different HAL targets and such, configuring devices for running tests on devices, ec.), so there wasn't much point in sharing code.In this initial rewrite, support for
dylibandvmvxbackends are added. CUDA and Vulkan are TODO.The "external config" functionality didn't provide enough flexibility -- putting appropriate framework functionality in a library and letting the downstream "own main()" works much better.
The current status is:
dylib: 367 pass / 61 failvmvx: 307 pass / 121 failFull list of failing tests are at: https://gist.github.com/silvasean/33663d18f27d6c40d70023e92a2874aa
And a rough breakdown of the dylib failure modes is available at: https://gist.github.com/silvasean/b5a14a4cf3dd2b70332e7efbec655d72
There seem to be about 10-ish issues (not all of them in IREE -- some are improvements to our IREE test config to properly handle multiple results/etc.).
The IREE bugs needed to pass all the Torch-MLIR E2E tests are being tracked in: https://github.com/google/iree/projects/44