[Feature Request - From Grayskull UXR] Operators have their dedicated library, similar to cuDNN

[adifrancescoTT]

Is your feature request related to a problem? Please describe. Right now I have to copy the operators out of TT-NN

Describe the solution you'd like I'd like the operators get their own library instead of tt-nn, like cuDNN, you just call and do something with them;

[ayerofieiev-tt]

TTNN is like cuDNN, operations are inside of it.

tt-metal repo contains:

• low level Metal staff
• TTNN lib

TTNN is a set of operations and infra to run them on top of metal. We think for now this is a reasonable breakdown.

[marty1885]

I believe this is my feedback during my call with Antonio. I'll add some contexts that I think changes things

@ayerofieiev-tt I am aware of the TTNN library. However using TTNN in 3rd party projects is very difficult. First, the TTNN library depends on Python and Pybind11. This is not acceptable for projects like GGML where dependencies must be minimal (and ideally not depending on another language besides C and C++).

https://github.com/tenstorrent/tt-metal/blob/main/ttnn/CMakeLists.txt#L41

However the actual ttnn_lib that has the actual kernels is built as an object file. Plus the headers are berried deep within the metal repo. Making it difficult to integrate into other projects.

https://github.com/tenstorrent/tt-metal/blob/main/ttnn/CMakeLists.txt#L16

It would be ideal if ttnn_lib could be isolated as it's own library and make outside linkage possible (without going through Python). Or better, TTNN keeps on being the Python library. But a new ttDNN is created to be the universal operator library.

[ayerofieiev-tt]

@marty1885 thank you for adding details. All fair. Reopened.

Your expectations are aligned with ours. I expect to follow up next week with a plan.

[davorchap]

@marty1885 thank you for adding details. All fair. Reopened.

Your expectations are aligned with ours. I expect to follow up next week with a plan.

Thanks!