Closed nv-hwoo closed 2 months ago
Would you be able to look into what the most Pythonic way of doing this is? It seems to be that having some functionality be under a subcommand and some functionality would not be the generally suggested approach. I'd also want to see if this messes up the help menu.
Great point. Yeah I am not really satisfied on how the current command/subcommand is structured, and we should definitely look into this (and I think @debermudez has already started the conversation on this). But I think that effort will require more effort than this current PR. I have TMA-1900 ticket to refactor CLI for this.
Add
compare
subcommand that allows users to generate plots to compare multiple runs.Users can provide paths to profile export files using
--files
CLI option as followingwhich will generate a set of plots using these runs as well as pre-filled, default yaml configuration file that users can edit to tweak the plotting parameters.
Users can also directly pass the yaml config file to generate customized plots detailed in the yaml file as following
General workflow of user can be:
genai-perf compare --files [file, ...]
that gives plots as well as the initial yaml config filegenai-perf compare --config [YAML file]
by tweaking plotting parameters such as title, x and y labels, x and y metrics, and etc.