search

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.85k stars 618 forks source link

[GPU] Long compilation time/excessive ops generated #18568

Closed IanWood1 closed 2 weeks ago

IanWood1 commented 2 months ago

Some multi-reduction dispatches take a long time to compile. For context, https://github.com/iree-org/iree/issues/18479 identifies numerical issues with the current pipeline and https://github.com/iree-org/iree/pull/18519 should solve this issue. But the compilation time for the 'good dispatch' linked on #18479 is quite long (for me about 40 seconds). IR dumps show that a bunch of ops being generated during LLVMGPUVectorDistribute.

To Reproduce:

  1. Checkout IREE at 2047f61(from Bangtian's PR)
  2. Download the mlir file https://gist.github.com/IanWood1/add7f516d88b81ce8b3d819bf39c2025
  3. Run iree-compile good-source2.mlir -o /dev/null --iree-hal-target-backends=rocm --iree-dispatch-creation-enable-aggressive-fusion

Additionally, following the same steps above but using main (tested at 914858fb89c028a94564b590626aab519d611e54) also exhibits long (~20 seconds) compilation times. This appears to be an unrelated (and there is a reasonable amount of ops but some very big vector.shuffle masks) since they are going down different pipelines. This should be resolved after switching pipelines.

ScottTodd commented 2 months ago

Tips for debugging: https://iree.dev/developers/debugging/compile-time-regressions/

IanWood1 commented 2 weeks ago

https://github.com/iree-org/iree/issues/18479