Ring all gather for in0 in matmul 1D

[johanna-rock-tt]

Problem description Mcasting m0 for matmul1D is a bottleneck due to NOC congestion.

Proposed solution Ring all gather in0, where in each step of the matmul computation the new local chunk is processed using the corresponding weight chunk and in parallel the local chunk is sent to the next node in the ring.

[johanna-rock-tt]

Initially: activation sharded across 24 cores

1. Compute on local chunk and corresponding weight chunk (need to know step id), and accumulate to output
2. Send local chunk point to point along ring (of 24 cores) to next core

Repeat at 2.

[avoraTT]

Let's assume in1 is interleaved in DRAM at the moment?

[johanna-rock-tt]

Yes, let's assume in1 is in DRAM, and for measuring perf we can comment in1 so that we mimic it being already locally on the cores in the correct format/order.

[johanna-rock-tt]

Let's collect all information, assumptions, and plans for the implementation here.