FlagOpen / FlagGems

FlagGems is an operator library for large language models implemented in Triton Language.
Apache License 2.0
347 stars 48 forks source link

Add Script to Calculate Summary Information for Benchmark Results #271

Closed kiddyjinjin closed 3 weeks ago

kiddyjinjin commented 4 weeks ago

Purpose

This PR introduces a new script to calculate summary information for benchmark results, aimed at providing insights into the average speedup for each operation categorized by data type.

Changes Made

Usage

To generate summary information from benchmark logs, run the following command:


python summary_for_plot.py path_to_your_log_file.log
kiddyjinjin commented 3 weeks ago

I suggest adding a shell script to ensure the order of operator benchmarks.

Since the command for generating benchmarks is quite flexible (it might specify op_name, a specific file, or simply run all operations without any specification), it would be challenging to pre-sort the operations to be benchmarked. However, sorting the operations in the benchmark results after the run is a great suggestion.