TTNN Op Interface: Op Perf model

[rjakovljevicTT]

What Provide a TTNN API to return Op perf estimate to the TT-MLIR optimizer. The goal of this issue is to do an end-to-end PoC and conclude what would be the proper solution. Based on the conclusion we will open a new issue to define the whole work.

PoC - September '24

• [ ] Select few ops for experiment
• [ ] For the selected ops, measure compute perf, establish analytical solution, compare the two
• [ ] For the selected ops, measure data movement perf, establish analytical solution, compare the two

PoC Initially, on an example of MNIST NN and all Ops that are parts of MNIST, do the following tasks:

• [ ] Implement Op perf API in TTNN (we should be in sync with core TTNN team),
• [ ] Consume the API in TT-MLIR Optimizer.
• [ ] Conclude on the Op perf API design in TTNN and define the issue to do the whole work on TTNN side.

The above tasks should be done Iteratively until it works well for both TT-MLIR Optimizer and TTNN.

[rjakovljevicTT]

We know it would be a lot of complex work to provide and maintain "Op perf" API with Ops being actively developed and optimized, hence we should also explore potential alternatives in terms of how Optimizer would make performance related decisions. This is to be discussed at least with @s-jovic, @pavlepopovic, @nobradovictt.

[mbezuljTT]

For this sprint, we are not implementing proposed POC.

303 and #304 have been flagged as more urgent, and they carry similar challenges as #305:

• API integration, how to avoid bloating op and building all details without compiler being too specific about the op variant and every single parameter.
• Maintenance, how to avoid having maintenance hell

On the perf estimate itself, we do want to make a set of experiments to establish a method to measure/calculate perf of op (compute and data movement). Captured under PoC - September '24