sharkdp / hyperfine

A command-line benchmarking tool
Apache License 2.0
21.67k stars 349 forks source link

Add short flags for `--export` #691

Open Neved4 opened 12 months ago

Neved4 commented 12 months ago

Currently, the --export variants lack short flags, forcing commands to be verbose. This issue is intended to spark discussion rather than to provide a fixed change.

Inspired by the approach used in pandoc, we could introduce a short flag, -e or -f, as in -f <FORMAT> to reduce repetition. The specific letter isn't crucial, consistency is. Here's an example:

Current Addition
--export-asciidoc
--export-csv
--export-json
--export-markdown
--export-orgmode
-f asciidoc
-f csv
-f json
-f markdown
-f orgmode
This would align with existing flags, such as --sort auto and --output null.

There are multiple ways to implement this addition. For example, we could pass a second argument -f asciidoc <FILE>, or use it in combination with a new option -f asciidoc -o <FILE>.

This also opens the possibility for hyperfine to default to a specific format based on the file extension. Namely, -o file.md would be equivalent to --export-asciidoc <FILE>, simplifying the command further. That default could be overriden by specifying the format explicitly: -f asciidoc -o file.md.

That strategy could also be used with --parameter flags, and it may break compatibility:

Current Addition
--parameter-scan
--parameter-step-size
--parameter-list
-P scan
-P step-size
-P list

Ultimately the maintainers know what's best for the project, make any adjustments you see fit 🚀

sharkdp commented 12 months ago

Thank you for writing this down. I like the idea!

How would we export to multiple formats? I sometimes export to markdown and json at the same time, for example.

Neved4 commented 11 months ago

Thanks for the feedback! You've raised an excellent point.

Let's say we're targeting all export formats. Currently, we do that with:

hyperfine \
    --export-asciidoc FILE.asciidoc \
    --export-csv FILE.csv           \
    --export-json FILE.json         \
    --export-markdown FILE.md       \
    --export-markdown FILE.org

With an auto-detection feature in place, we could rely on default file extensions and implicit file format inference:

hyperfine -f FILE.asciidoc -f FILE.csv -f FILE.json -f FILE.md -f FILE.org

We can also reduce verbosity by adding the ability to group formats:

hyperfine -f FILE.asciidoc,FILE.csv,FILE.json,FILE.md,FILE.org

Another option is to reuse the existing idea of the parameter list -L, and introduce an -E flag for exporting formats. For example:

hyperfine -E asciidoc,csv,json,md,org '-f FILE.{ext}'

Consider a different scenario, where we want to export all formats file extensions arbitrarily. Right now:

hyperfine \
    --export-asciidoc result.org  \
    --export-csv result.asciidoc  \
    --export-json result.csv      \
    --export-markdown result.json \
    --export-orgmode result.md

With a -E flag we have the option to combine with an hypothetical -F:

hyperfine \
    -F fmt asciidoc,csv,json,markdown,orgmode \
    -E ext org,asciidoc,csv,json,md \
    '-f {fmt} result.{ext}'
Note that in flags -E and -F, quoted blocks will be interpreted exclusively for exporting, not adding extra commands to be benchmarked.

In the long run, this level of parametrization may not be high priority, as it appears to cater more to rare use cases.