tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
396 stars 49 forks source link

Perf_Mon: Update OP profiler and post processing script #1594

Open mo-tenstorrent opened 1 year ago

mo-tenstorrent commented 1 year ago

Follow the suggested headers from the spreadsheet.

https://tenstorrent.sharepoint.com/:x:/r/sites/Jasmina/_layouts/15/Doc.aspx?sourcedoc=%7B10AF97D8-38F7-4A56-9B2F-1F78FE7F92C0%7D&file=reports%20-%20brainstorming.xlsx&action=default&mobileredirect=true

  1. Opcode: Modify OP infra to set name as OP code
  2. OP Type: Add a OP_profiler setter and pybind to be set by OP infra and python
  3. Opcode Call Count: Add a OP_profiler setter to be set by OP infra
  4. OP Cache Setting: Add a OP_profiler setter to be set by OP infra
  5. OP Cache Hit: Add a OP_profiler setter to be set by OP infra
  6. OP Cache Hash: Add a OP_profiler setter to be set by OP infra
mo-tenstorrent commented 1 year ago

@jvasilje @TT-billteng @tt-aho @arakhmati

Feel free to comment

TT-billteng commented 1 year ago

Should OP Cache Hash be part of the metadata field? Fallback ops won't have a hash right?

mo-tenstorrent commented 1 year ago

Good call, not a fan of NAs or dashes in the field?

TT-billteng commented 1 year ago

Good call, not a fan of NAs or dashes in the field?

Fallback OPs don't have an OP cache hash so that's why I'm wondering

arakhmati commented 1 year ago

Yeah, I don't think Op Cache Hash should be part of the metadata. Especially, because eventually I think the hash could become an integer instead of a string

mo-tenstorrent commented 1 year ago

Test files that ran and passed against https://github.com/tenstorrent-metal/tt-metal/pull/1615

profile_log_ops_stable_diff_1pass.tgz profile_log_ops_small_mock.tgz profile_log_ops_small_mock.csv profile_log_ops_full_bert.csv profile_log_ops_stable_diff_1pass.csv profile_log_ops_full_bert.tgz