search

tenstorrent / tt-mlir

Tenstorrent MLIR compiler
https://tenstorrent.github.io/tt-mlir/
Apache License 2.0
51 stars 7 forks source link

Support for conv2d op for tt-nn backend #440

Open nvukobratTT opened 3 weeks ago

nvukobratTT commented 3 weeks ago

Variants of convolutions that we want to support:

nvukobratTT commented 2 days ago

@LPanosTT is this issue ready to be closed? Or some of the mentioned convs are still required? If this is still WIP, let's update issue desc with latest details

LPanosTT commented 2 days ago

@nvukobratTT For larger convolutions this op still runs into issues.

For example, a convolution of a tensor of shape (1, 512, 512, 1) with a 3x3 kernel, one output channel, and stride of 1 will yield this error:

Statically allocated circular buffers on core range [(x=0,y=0) - (x=7,y=7)] grow to 3259296 B which is beyond max L1 size of 1499136 B

If the kernel is changed to be 4x4, the error thrown is: 

Out of Memory: Not enough space to allocate 16779264 B L1 buffer across 3 banks, where each bank needs to store 5593088 B

Both of these occur in runtime when executing the TTNN op.

As far as I can find, larger convolutions will always cause one of these two errors. Some you may be able to get through by using bfloat16 instead of float32.

nvukobratTT commented 1 day ago

Great! Thanks for the details!

Let's keep this issue open until we determine will be an issue for ResNet 50 as well. If not, we can transfer this issue to a separate milestone and explore it further.