ordo-one / package-benchmark

Swift benchmark runner with many performance metrics and great CI support
Apache License 2.0
326 stars 25 forks source link

`benchmark init` plugin command fails #218

Closed finestructure closed 10 months ago

finestructure commented 10 months ago

I'm trying to get started with the plugin-based setup

swift package --allow-writing-to-package-directory benchmark init MyNewBenchmarkTarget

but I'm getting an error:

❯ swift package --allow-writing-to-package-directory benchmark init CheckRedirectsBenchmark
Building for debugging...
Build complete! (0.58s)
Build complete!
Building BenchmarkTool in release mode...
Building benchmark targets in release mode for benchmark run...
Error: Missing expected argument '--targets <targets> ...'
Help:  --targets <targets>  The targets
Usage: benchmark-tool [<options>] --targets <targets> ... --command <command> --format <format> --baseline-storage-path <baseline-storage-path> --grouping <grouping>
  See 'benchmark-tool --help' for more information.
One or more benchmarks returned an unexpected return code 16384
error: benchmarkUnexpectedReturnCode

Next I tried with --targets, think it might be docs drift:

❯ swift package --allow-writing-to-package-directory benchmark init --targets CheckRedirectsBenchmark
Building for debugging...
Build complete! (0.58s)
Unknown option/flag specfied: ["--targets"]
error: invalidArgument

I tried showing help but that fails the same way:

❯ swift package --allow-writing-to-package-directory benchmark init --help
Building for debugging...
Build complete! (0.57s)
Unknown option/flag specfied: ["--help"]
error: invalidArgument

I've also tried benchmark --targets CheckRedirectsBenchmark init and benchmark --targets CheckRedirectsBenchmark --command init which seemed sensible looking at the initial error message but they also fail with invalidArgument.

Did I mess something up setting this up? The only change I've made is add

        .package(url: "https://github.com/ordo-one/package-benchmark.git", from: "1.21.1"),

to my package manifest.

I'll try the manual setup, just wanted to leave a note that there might be a problem with the "happy path" :)

hassila commented 10 months ago

Thanks a lot for the report, this fell between the chairs during previous refactoring - will be fixed by #219

hassila commented 10 months ago

@finestructure please give https://github.com/ordo-one/package-benchmark/releases/tag/1.21.2 a go and it should now work as expected, thanks again.

hassila commented 10 months ago

So documentation should be ok now, you should be able to do:

swift package --allow-writing-to-package-directory benchmark init CheckRedirectsBenchmark

hassila commented 10 months ago

or the slightly easier to remember:

swift package --disable-sandbox benchmark init CheckRedirectsBenchmark

finestructure commented 10 months ago

That works now, thanks a lot for the quick fix!