search

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.85k stars 620 forks source link

[LLVMCPU] Migrate to TileRootAndFuseProducerConsumer pipeline #19163

Closed pashu123 closed 21 hours ago

pashu123 commented 1 week ago

Starting with mmt4dTilingExpert pipeline.

hanhanW commented 1 week ago

It actually does nothing for the all existing tests and compilation passes. Can we add a pipeline test that has dequant ops + mmt4d?

pashu123 commented 3 days ago

It actually does nothing for the all existing tests and compilation passes. Can we add a pipeline test that has dequant ops + mmt4d?

I'll disable this and add tests https://github.com/iree-org/iree/blob/b68c535ece28e139492606f391493f3e95242420/compiler/src/iree/compiler/DispatchCreation/FormDispatchRegions.cpp#L546

hanhanW commented 3 days ago

It actually does nothing for the all existing tests and compilation passes. Can we add a pipeline test that has dequant ops + mmt4d?

I'll disable this and add tests

https://github.com/iree-org/iree/blob/b68c535ece28e139492606f391493f3e95242420/compiler/src/iree/compiler/DispatchCreation/FormDispatchRegions.cpp#L546

I thought that is for consumer fusion (i.e., mmt4d + elementwise,etc)? And what you're looking for is producer fusion? We don't need to enable it on e2e flow for now, because I think we'll hit some more issues. To incrementally support it, you can add a pipeline test for now. E.g., https://github.com/iree-org/iree/blob/23c32c633c01e0237cf5f3815b6647cf01827832/compiler/src/iree/compiler/Codegen/LLVMCPU/test/pipeline_tests.mlir#L259-L276

pashu123 commented 3 days ago

It actually does nothing for the all existing tests and compilation passes. Can we add a pipeline test that has dequant ops + mmt4d?

I'll disable this and add tests https://github.com/iree-org/iree/blob/b68c535ece28e139492606f391493f3e95242420/compiler/src/iree/compiler/DispatchCreation/FormDispatchRegions.cpp#L546

I thought that is for consumer fusion (i.e., mmt4d + elementwise,etc)? And what you're looking for is producer fusion? We don't need to enable it on e2e flow for now, because I think we'll hit some more issues. To incrementally support it, you can add a pipeline test for now. E.g.,

https://github.com/iree-org/iree/blob/23c32c633c01e0237cf5f3815b6647cf01827832/compiler/src/iree/compiler/Codegen/LLVMCPU/test/pipeline_tests.mlir#L259-L276

I see, it makes sense.