Describe the bug
The --kernelVerbose flag overwrites full kernel names generated by rocprof. In the profile step, this results in having to add an argument to be able to analyze with full kernel names, but omniperf analyze also accepts and respects --kernelVerbose and results in overwriting the file containing kernel names. It seems the only way to recover the longer names is to re-run omniperf profile with the appropriate level of --kernelVerbose.
Development Environment:
Linux Distribution: Various
Omniperf Version: 1.0.10, 1.1.0-PR1
GPU: MI200
Cluster (if applicable): Thera
To Reproduce
Steps to reproduce the behavior:
tar -xzvf omniperf_verbose_repro.tar.gz
cd omniperf_verbose_repro
git clone git@github.com:kokkos/kokkos.git
cd build
./build.sh
make -j64
omniperf profile -n kokkos_dot --kernelVerbose=5 -- ./dot
omniperf analyze -p workloads/kokkos_dot/mi200 --list-kernels
(see full-length kernel names)
omniperf analyze -p workloads/kokkos_dot/mi200 --kernelVerbose=1 --list-kernels
(see shortened kernel names)
omniperf analyze -p workloads/kokkos_dot/mi200 --kernelVerbose=5 --list-kernels
(see shortened kernel names, instead of full-length kernel names)
Expected behavior
Since omniperf analyze already respects the --kernelVerbose argument, the shortened kernel names should be stored in a copy of the csv that holds the full kernel names. This way, omniperf profile will return full kernel names and shortened kernel names by default, and omniperf analyze can interpret the --kernelVerbose argument to find the file with the appropriate level of shortening (or create it if necessary).
More generally, the full profiling data obtained from rocprof shouldn't be modified by omniperf analyze. Any modifications to the data should be made in such a way that the change can be reverted.
coleramos425
commented
7 months ago
Describe the bug The
--kernelVerboseflag overwrites full kernel names generated by rocprof. In the profile step, this results in having to add an argument to be able to analyze with full kernel names, butomniperf analyzealso accepts and respects--kernelVerboseand results in overwriting the file containing kernel names. It seems the only way to recover the longer names is to re-runomniperf profilewith the appropriate level of --kernelVerbose.Development Environment:
To Reproduce Steps to reproduce the behavior: tar -xzvf omniperf_verbose_repro.tar.gz cd omniperf_verbose_repro git clone git@github.com:kokkos/kokkos.git cd build ./build.sh make -j64 omniperf profile -n kokkos_dot --kernelVerbose=5 -- ./dot omniperf analyze -p workloads/kokkos_dot/mi200 --list-kernels (see full-length kernel names) omniperf analyze -p workloads/kokkos_dot/mi200 --kernelVerbose=1 --list-kernels (see shortened kernel names) omniperf analyze -p workloads/kokkos_dot/mi200 --kernelVerbose=5 --list-kernels (see shortened kernel names, instead of full-length kernel names)
Expected behavior Since
omniperf analyzealready respects the--kernelVerboseargument, the shortened kernel names should be stored in a copy of the csv that holds the full kernel names. This way,omniperf profilewill return full kernel names and shortened kernel names by default, andomniperf analyzecan interpret the--kernelVerboseargument to find the file with the appropriate level of shortening (or create it if necessary).More generally, the full profiling data obtained from rocprof shouldn't be modified by
omniperf analyze. Any modifications to the data should be made in such a way that the change can be reverted.Additional context omniperf_verbose_repro.tar.gz