[EPIC][CPU] Enable predictable performance on mixed-types GEMM using data-tiling

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.

http://iree.dev/

Apache License 2.0

2.51k stars 558 forks source link

[EPIC][CPU] Enable predictable performance on mixed-types GEMM using data-tiling #15629

Open hanhanW opened 8 months ago

hanhanW commented 8 months ago

This EPIC tracks all the related work.

Tasks related to core functionality

Tasks

Tasks related to performance improvements

[x] https://github.com/openxla/iree/pull/15585
[ ] https://github.com/openxla/iree/issues/15554
[x] https://github.com/openxla/iree/pull/15651
[ ] https://github.com/openxla/iree/issues/15441
[x] linalg.matmul_transpose_b is not data-tiled
[x] https://github.com/openxla/iree/issues/15600
[ ] https://github.com/openxla/iree/issues/15611

llama2 specific tasks

Tasks

[ ] https://github.com/openxla/iree/issues/15566
[ ] https://github.com/openxla/iree/issues/15195
[ ] Enable e2e integration tests

hanhanW commented 8 months ago

@Max191 I think you have some local patches and ideas that are required for mixed-types data-tiling work, could you add them to tasklist accordingly?

@bjacob please help update this if there are on-going/TODO tasks in your mind.

@MaheshRavishankar I created an epic to help us understand better what needs to be done for mixed-types data-tiling, and the work we've been working on.

Thank you all for all the awesome work!

hanhanW commented 8 months ago

For small tasks, adding a brief description to tasklist is good enough. For large tasks, it would be good if you can create an issue/epic. It's not necessary to do it now, but please help add a brief description. Thank you!