Setup valgrind/callgrind

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

Apache License 2.0

396 stars 49 forks source link

Setup valgrind/callgrind #10918

Open eyonland opened 1 month ago

eyonland commented 1 month ago

It would be helpful to have an additional tool to verify performance. valgrind/callgrind are well-known tools that help profiling the host code. Ideally this would be setup as a github workflow to make it easy for anyone to run these reports and use this to track down performance issues.

patrickroberts commented 1 week ago

@TT-billteng @tt-rkim @ttmchiou hey, I'm interested in setting up a manually triggered workflow in tt-metal to automate running some of the targets with callgrind and generating the call graph diagrams as artifacts. Do you think we could schedule some time to pin down the details for this work and map out a plan?

tt-rkim commented 1 week ago

Sure, we should probably figure out:

if this is scheduled as nightly or manually called, and if manually called, how often do people want to call this workflow?
How many and what kind of build resources will this take?
And related to the above, do we have enough resources?

I've put this as P3 on the infra team board.