search

bencherdev / bencher

🐰 Bencher - Continuous Benchmarking
https://bencher.dev
Other
536 stars 25 forks source link

Treat hyperfine parameterized benchmarks as separate benchmarks #472

Open epompeii opened 2 months ago

epompeii commented 2 months ago

Currently, hyperfine parameterized benchmarks are all given the same benchmark name.

Parameterized benchmarks should include the parameter name and value in their Bencher benchmark name. They may also be set as tags, once that is implemented: https://github.com/bencherdev/bencher/issues/240

epompeii commented 2 months ago

Correction, parameterized benchmarks are all given the same name when the --command-name option is used:

-n, --command-name Give a meaningful name to a command. This can be specified multiple times if several commands are benchmarked.

See this comment for an example: https://github.com/bencherdev/bencher/issues/471#issuecomment-2265364452

The parameters are available in the JSON output:

"parameters": {
  "time": "0"
}