tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
466 stars 73 forks source link

Performance Sheet Analysis Tool #12468

Open vshenoyTT opened 1 month ago

vshenoyTT commented 1 month ago

Performance Sheet Analysis Tool

Intro

This is a web tool for easily analyzing Excel and CSV performance sheets. Primarily, it calculates FPS and Adjusted Utilization for each operation.

GitHub Repository Deployment (available till 9/13)

Codebase

The tool is built using Streamlit for the interface, with Pandas and NumPy for data manipulation and sheet formatting. The main file containing the web interface and data manipulation is streamlit_app.py. The dependencies are listed in requirements.txt, and the .streamlit file contains configurations for UI styling.

Usage

How it Works

To utilize the tool, a user selects the Grayskull or Wormhole configuration, adjusting core count for calculations accordingly. Then, the user uploads a .csv or .xlsx performance sheet. The tool automatically starts the analysis process

Screenshot 2024-09-05 at 12 17 33 PM

Results

After a few seconds, the tool displays FPS for All Operations, MatMul + Conv Operations only, and Other On-Device Operations (excluding MatMul + Conv). The user can download performance sheets filtered for any of these configurations.

Screenshot 2024-09-05 at 12 17 50 PM

Downloaded Spreadsheet

If downloaded, the modified spreadsheet appends an Adjusted Utilization column for each operation, and an FPS column for the configuration. It also shuffles the Core Count + Device Kernel Duration columns to the end for easy comparison.

Screenshot 2024-09-10 at 10 50 10 AM

Run Locally

To run the tool locally, install the requirements from requirements.txt using the command pip install -r requirements.txt. Then, within the directory, run the command streamlit run streamlit_app.py. The web tool will default to opening at http://localhost:8501/, unless port 8501 is already in use.

vshenoyTT commented 1 month ago

This tool also has a complementary web tool to visually graph operations.

GitHub Repository Deployment (available till 9/13)

Codebase

The tool is built using Streamlit for the interface, along with Pandas, NumPy, and MatPlotLib for data manipulation and graphing. The main file containing the web interface and data manipulation is streamlit_app.py. The dependencies are listed in requirements.txt, and the .streamlit file contains configurations for UI styling.

How it Works

To utilize the tool, a user selects the Grayskull or Wormhole configuration, adjusting core count for calculations accordingly. Then, the user uploads a .csv or .xlsx performance sheet. The tool automatically starts the graphing process

Screenshot 2024-09-10 at 11 20 29 AM

Results

After a few seconds, the tool displays multiple graphs for the categories of All Operations, MatMul + Conv Operations only, and Other On-Device Operations (excluding MatMul + Conv). Each category has graphs for Core Count + Utilization, Device Kernel Duration + Utilization, and Device Kernel Duration vs Utilization.

Screenshot 2024-09-10 at 11 19 46 AM Screenshot 2024-09-10 at 11 21 20 AM

There is also a Pie Chart to show the breakdown of Operation Types in the data.

Screenshot 2024-09-10 at 11 19 35 AM

Downloaded Spreadsheet

Graphs and data are downloadable via buttons below each graph and a Download Data section at the bottom of the tool.

Run Locally

To run the tool locally, install the requirements from requirements.txt using the command pip install -r requirements.txt. Then, within the directory, run the command streamlit run streamlit_app.py. The web tool will default to opening at http://localhost:8501/, unless port 8501 is already in use.

mbahnasTT commented 1 month ago

fyi @davorchap @jvasilje @eyonland @ayerofieiev-tt @dvartaniansTT