perftool-incubator / rickshaw

A project to facilitate execution of benchmarks and tools via extensions for many target environments
Apache License 2.0
0 stars 10 forks source link

Support the ability for extra benchmark params to be detected based on sample result #524

Open atheurer opened 3 months ago

atheurer commented 3 months ago

There are some benchmarks which have a very large number of possible params, of which the defaults may not be known (or they might change) or these params are defined by the benchmark based on other input. These params are not always provided by the user when doing a run, but these params need to be known for comparison of results later.

After post-processing of a sample is complete, rickshaw should check for an optional param-script for the benchmark, and if defined, run it with the same params as done with the benchmark post-process script. A json of params should be generated, and these should be added by rickshaw-run to the run object (and what becomes rickshaw-run.json).