tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
391 stars 47 forks source link

Extend Reshape op to support Row_major Layout #2925

Closed Sudharsan-V closed 4 months ago

Sudharsan-V commented 11 months ago

The current state of the TT Metal reshape operation allows for the reshaping of tensors only when the tensor is in the TILE Layout.

The request is to extend this capability to enable the TT Metal reshape operation to function even when the tensor is in a Row major layout.

We have a use case where the tensor needs to be reshaped. But,tt_lib.tensor.reshape can function only when the tensor is in the TILE layout.

tt-aho commented 11 months ago

We have support for RM reshape, but it is fairly limited. What are your original and target reshape sizes?

Sudharsan-V commented 11 months ago

Original shape: [1, 1, 60, 768] Target shape: [1, 60, 12, 64]

muthutt commented 10 months ago

Same issue encountered with Mistral model bringup where most of the reshapes are done on CPU for various reasons:

VirdhatchaniKN commented 10 months ago

Extended for a row-major layout, and also for values of HW that are not multiples of 32

Tested for the following shapes

- [4, 4, 32, 32] to [-1, 2, 32, 32],
- [4, 4, 32, 32] to [2, -1, 32, 32],
- [4, 4, 32, 32] to [4, 1, -1, 32],
- [4, 4, 32, 32] to [4, 2, 32, -1],
- [2, 3, 4, 4] to [2, 12, 1, 4],
- [1, 3, 6, 4] to [3, 1, 3, 8],
- [1, 1, 60, 768] to [1, 60, 12, 64],
- [2, 4, 128, 4] to [1, 2, 64, 32],
- [1, 11, 64, 2] to [1, 11, 1, 128],
- [1, 2, 64, 32] to [2, 4, 128, 4],
- [3, 2, 2, 32] to [1, 3, 1, -1]
mtairum commented 4 months ago

Closing this issue as stale since many updates were made since then for reshape op. Also both Mistral and Mixtral bringup has been integrated on main without this issue.