search

nod-ai / iree-amd-aie

IREE plugin repository for the AMD AIE accelerator
Apache License 2.0
46 stars 23 forks source link

merge kernels in existing XCLBIN #418

Open nirvedhmeshram opened 3 weeks ago

nirvedhmeshram commented 3 weeks ago

This PR uses a new xclbin generation tool from https://github.com/Xilinx/mlir-aie/pull/1561 that allows us to merge kernel PDIs into the same XCLBIN. A mlir-aie wheel bump is needed to use the tool. An e2e example which forms multiple (three) dispatches is added.

nirvedhmeshram commented 2 weeks ago

Very cool that the e2e test works! Just to confirm: these matmuls form 3 different dispatches?

Yes that is correct.

nirvedhmeshram commented 2 weeks ago

On further investigating I realized that with the wheel the CI is using, no actual merging is taking place, you can see this error in the CI test added

Expected<T> must be checked before access or destruction.
Expected<T> value was in success state. (Note: Expected<T> values in success mode must still be checked prior to being destroyed).

This causes the merging to fail and it goes through the regular new xclbin per dispatch path. This is not what I see when I build my own mlir-aie. However, given I am seeing same performance with and without the merging achieved from this PR I think I will hit pause on this for now and come back to it later perhaps after this PR lands, https://github.com/nod-ai/iree-amd-aie/pull/422