microsoft / DeepSpeedExamples

Example models using DeepSpeed
Apache License 2.0
5.83k stars 990 forks source link

Update throughput-latency plot script #881

Closed lekurile closed 2 months ago

lekurile commented 3 months ago

This PR updates the plot_th_lat.py throughput-latency plot generation script to remove the concept of a backend (aml, fastgen, vllm) and generalize for any result output directory, irrespective of where it was run.

The PR also introduces the concept of an optional plot_config.yaml that resides within each result directory and allows for overrides in the plot formatting. An example config file may look like this:

label: "vLLM"
color: "purple"
marker: "o"
linestyle: "--"
polyfit_degree: 0
x_max : 30
y_max : 10

Each of the config parameters is optional, allowing for override of only the specific plot aspects required, however all parameters may also be provided.

A few nuances for the polyfit_degree and x/y_max parameters:

An example command executing the script may look something like this:

python3 src/plot_th_lat.py --data_dirs ./results/results-* --model_name <plot_model_title>

Or each result directory can be enumerated explicitly:

python3 src/plot_th_lat.py --data_dirs ./results/results-1 ./results/results-2 ./results/results-3 --model_name <plot_model_title>