bert model view to ttnn.reshape issues

[kevinwuTT]

bert model calls aten.view with these shapes:

input_shape -> output_shape:

essentially the same as ttnn.squeeze with dim = 0, compatible with ttnn.reshape [1, 16, 256, 256] -> [16, 256, 256] [1, 16, 256, 64] -> [16, 256, 64] [1, 16, 64, 256] -> [16, 64, 256] [1, 256, 4096] -> [256, 4096] [1, 256, 1024] -> [256, 1024]

essentially the same as ttnn.unsqueeze_to_4D, compatible with ttnn.reshape [16, 256, 256] -> [1, 16, 256, 256] [16, 256, 64] -> [1, 16, 256, 64]

unsqueeze, compatible with ttnn.reshape [256, 2] -> [1, 256, 2] [256, 1024] -> [1, 256, 1024] [256, 4096] -> [1, 256, 4096]

Not compatible with ttnn.reshape [1, 256, 1024] -> [1, 256, 16, 64] [1, 256, 16, 64] -> [1, 256, 1024]

[ayerofieiev-tt]

@mcw-zwakeelTT can you please add example unsqueeze?

[ayerofieiev-tt]

No known workarounds. Might be fixed around end of July.

[mcw-zwakeelTT]

@ayerofieiev-tt Unsqueezing last dim of 3D tensor to make the output a 4D. This throws a runtime error "Unable to reshape given tensor". (5, 2, 4) with dim 3 should produce (5, 2, 4, 1) but throws a runtime error. There is no such examples in bert test case though.

[ayerofieiev-tt]

I wonder if this works

input = T[1, 256, 1024]
shape = [1, 256, 16, 64]
reshape(input, layout) 

lower to

if(input is tile and output shape is not tiled)
  output = to_layout(input, RowMajor);  // [1, 256, 1024]
  output = reshape(output, shape); // [1, 256, 16, 64]
  output = to_layout(output, Tiled); // [1, 256, 16[32], 64]

Are ops in general (like softmax) aware about padding?

[mcw-zwakeelTT]

reshape to 4D is executed on device and only works if last dim is the same for both input & output. In your example it will throw a runtime error (1024 != 64) Currently reshape takes place after converting input tensor to row major layout. Tile layout is only valid if the tensor is reshaped to tiled height & width. The inputs of ops are padded if it is requested to be in tile layout.