search

pytorch-labs / tritonbench

Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.
BSD 3-Clause "New" or "Revised" License
24 stars 3 forks source link

Naming issue about profile reports #94

Open FindHao opened 13 hours ago

FindHao commented 13 hours ago

tritonbench export kineto traces to /tmp/tritonbench/<operator>/<debug_type>/<name>_<input>/<hostname>_<id>.pt.trace.json , ncu reports to /tmp/tritonbench/<operator>/<debug_type>/<name>_<input>/ncu_output.ncu-rep, and nsys reports to /tmp/tritonbench/<operator>/<debug_type>/<name>_<input>/nsys_output.nsys-rep. Because the report file names don't have any useful information, it is hard to identify different tests when we compare multiple reports.

Q1: Can we change the names with more information? Will this interrupt many places in testing?

Q2: Those reports locate at different folders for a single test, can we collapse the path structure like /tmp/tritonbench/\/debug_type_name_input_nsys_output.nsys-rep? and add mode(fwd,bwd,fwd_bwd) etc.?

Q3: Can we add username into the path? like /tmp/tritonbench/user or /tmp/tritonbench_user like torchinductor? This can help for multi-users servers' testing.

@xuzhao9

xuzhao9 commented 13 hours ago

Sure, all the changes make sense to me. We can add helper functions in path_utils.py to manage the output paths.