Emit correct tile_shapes for tensors

[jnie-TT]

Currently the tile_shape in the flatbuffer executable is hardcoded to 0x0. In runtime we want to use the tile_shape to determine if we want to tilize/untilize a tensor - some ops (like relu) can only run on tilized data.

We need compiler support that can emit the correct tile_shapes for input/output tensors.

[nsmithtt]

Hey everyone, we're already facing our first set of op constraints :). Since we don't have an interface with TTNN yet, perhaps there is an interim thing we could do?

TTIR has the op interface that provides getOperandConstraints which actually captures tilized vs row major. I'm thinking that there are a few options we could do in the short term:

• We could implement an analysis pass the (somehow) populates getOperandConstraints, i.e. some mechanism for the target backend to add their constraints on the TTIR ops.
  • Pros: Reuse the existing to_layout pass, would be agnostic and useful to both backends and runs before optimizer.
  • Cons: More complicated
• We could implement the same interface in TTNN and for now mark which ops in TTNN require tilized vs row major.
  • Pros: Less complicated
  • Cons: We have to reimplement the to_layout pass as to_memory_config pass, which will change the graph after the optimizer has run.

Let me know what you all think. We can schedule a follow up meeting if need be.

[mbezuljTT]

Hello,

I had a chat with @derdeljanTT and he reminded me our goal is to maintain cross-platform compiler. Today this means any linking to TTNN will break cross-platform compilation. While TTNN might be cross platform in the future, we should be careful about picking this dependency. @nsmithtt what do you think about it?

Going with analysis path means we have to hardcode op requirements manually.

Last time we have discussed op interface constraints, suggestion from TTNN folks was to build op constraints cache and establish straightforward way to rebuilding it. @nsmithtt are you aware of any development on this front or it's still up for a discussion?

I think we we have a cache of op constraints we could include it in the analysis path - this sounds easier than expanding TTNN.

I see this issue is blocking P0. @jnie-TT short-term is it reasonable to assume all ops can run on tilized data? if yes, are we ok to assume this on the compiler as well? It will come at perf cost.

Thanks Marko

[nsmithtt]

I think let's shoot for the op constraints cache (or config file) approach as that solves the cross platform issue too. @xanderchin, are you using this approach for ttnn sweep tests?