tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
https://docs.tenstorrent.com/ttnn/latest/index.html
Apache License 2.0
488 stars 80 forks source link

Add Tiny Tile support to CCL OPs #14517

Open jvegaTT opened 4 weeks ago

jvegaTT commented 4 weeks ago

The objective of this issue is to create the infrastructure needed and modify the implementations so that tiny tiles (i.e. tiles with a size smaller than 32x32) can be supported by the AllGather, Reduce-Scatter, and Barrier OPs. The objectives are below:

jvegaTT commented 3 weeks ago

Re-scoped to only modify tile size in the h direction to comply with hardware limitations.