perftool-incubator / crucible

A performance testing and analysis automation framework
Apache License 2.0
12 stars 7 forks source link

Add multibench support to all-in-one json #307

Closed rafaelfolco closed 12 months ago

rafaelfolco commented 1 year ago

This patch integrates the multibench functionality into the crucible run files, aka all-in-one json. It includes single bench and multibench run mode when --from-file is used. Backwards compatibility takes care of the old run mode, which sets the benchmark from the cli with 'crucible run <benchmark(s)> [args]'.

A dry run mode is added through the env var CRUCIBLE_DRY_RUN=1 to help testing debugging param construction, parsing and validation.