search

tenstorrent / tt-mlir

Tenstorrent MLIR compiler
https://tenstorrent.github.io/tt-mlir/
Apache License 2.0
59 stars 7 forks source link

[Ops] Transpose: Support for transpose over -3 dim when it's not tile-dim aligned #558

Open nvukobratTT opened 1 month ago

sdjordjevicTT commented 1 week ago

@nvukobratTT @mtopalovicTT do we need some action here?

nvukobratTT commented 1 week ago

@nvukobratTT @mtopalovicTT do we need some action here?

Yes, we'll need it.

Currently, we're using a TensorFlow variation of ResNet 50 to by pass the issue (a workaround). However, without transposing over -3 I'm not certain if can we push any conv-based model from PyTorch. More precisely, we need to transpose over -3 to convert channel first into channel last conv2d.

sdjordjevicTT commented 1 week ago

Do we have a repro MLIR for it? Do we need to create some issues on the metal side?

nvukobratTT commented 1 week ago

Do we have a repro MLIR for it? Do we need to create some issues on the metal side?

@LPanosTT will create an issue today for this problem on tt-metal repo with TT-NN as repro. I'm not sure do you see any value of MLIR repro as well, as we have TT-NN already? :))

sdjordjevicTT commented 1 week ago

Nope, I am just trying to understand do you need anything urgent from us regarding this ongoing issue? :)

nvukobratTT commented 1 week ago

Nope, I am just trying to understand do you need anything urgent from us regarding this ongoing issue? :)

No worries, I already discussed it with Mladen, this one is not P0 at the moment, but has the potential to become one in upcoming weeks :))

sdjordjevicTT commented 1 week ago

Great, thanks for clarification! :)

dgolubovicTT commented 1 week ago

Hi, I have experienced the need for transpose along -3 dimension while bringing up LLama3B. Namely, initial graph contains transposes along -3 dim:

image

Currently, those transposes are decomposed after optimization passes to following ops: 2xpad_tile, matmul, vslice, vstack and 2x narrow. I ran the transpose test changing one of the transpose axes to -3 and I get that:

image

Since this pad_tile is not supported end to end, and vslice and vstack are depricated, I would say that transpose along -3 is a blocker for further bringup of LLama 3B.

nvukobratTT commented 1 week ago

Thanks for the details @dgolubovicTT! This now became a blocker for Llama 3B, and near future blocker for ResNet 50.

@dgolubovicTT when you open up issues on tt-metal, please link it here as a ref 💪

sdjordjevicTT commented 5 days ago

Do we have an issue created on the metal side?

nvukobratTT commented 5 days ago

Do we have an issue created on the metal side?

https://github.com/tenstorrent/tt-metal/issues/13005

nvukobratTT commented 5 days ago

I'm updating blocker field as well