search

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
430 stars 59 forks source link

Remove ttl custom matmuls and replace with ttnn matmuls #8174

Closed TT-BrianLiu closed 3 months ago

TT-BrianLiu commented 5 months ago

Remove all custom matmuls/bmms and replace with ttnn matmuls/linears. There might be minor host perf degradation since some of the asserts are in python now.

List:

davorchap commented 5 months ago

Remove all custom matmuls/bmms and replace with ttnn matmuls/linears. There might be minor host perf degradation since some of the asserts are in python now.

List:

  • [x] bert matmuls/bmms

  • [ ] falcon matmuls/bmms

  • [ ] matmul_1d

  • [ ] Others...

With migration to C++ all of the asserts will be Program cached?

TT-BrianLiu commented 5 months ago

Asserts, getting program config, and ttnn tensor rank handling will eventually be cached (everything inside op struct essentially). But we need to review how we want to handle where this happens.

TT-BrianLiu commented 3 months ago

Tracked by this issue now: https://github.com/tenstorrent/tt-metal/issues/9492