Add an “scrape” mode that guesses grammar based on `cmd --help` output

[adaszko]

It should read the text on stdin and output the grammar on stdout so that it can be captured by the user and tweaked until it’s satisfactory. The idea behind this is to avoid necessary manual editing as much as possible.

It’s best to implement it in Rust so that no additional dependencies are necessary. Most likely using the nom crate, just like the grammar parser already does.

Example:

$ cargo test --help
Execute all unit and integration tests and build examples of a local package

Usage: cargo test [OPTIONS] [TESTNAME] [-- [args]...]

Arguments:
  [TESTNAME]  If specified, only run tests containing this string in their names
  [args]...   Arguments for the test binary

Options:
  -q, --quiet                   Display one character per test instead of one line
      --lib                     Test only this package's library unit tests
      --bin [<NAME>]            Test only the specified binary
      --bins                    Test all binaries
  -v, --verbose...              Use verbose output (-vv very verbose/build.rs output)
      --example [<NAME>]        Test only the specified example
      --color <WHEN>            Coloring: auto, always, never
      --examples                Test all examples
      --frozen                  Require Cargo.lock and cache are up to date
      --test [<NAME>]           Test only the specified test target
      --locked                  Require Cargo.lock is up to date
      --tests                   Test all tests
      --bench [<NAME>]          Test only the specified bench target
      --offline                 Run without accessing the network
      --benches                 Test all benches
      --config <KEY=VALUE>      Override a configuration value
      --all-targets             Test all targets
  -Z <FLAG>                     Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details
      --doc                     Test only this library's documentation
      --no-run                  Compile, but don't run tests
      --no-fail-fast            Run all tests regardless of failure
  -p, --package [<SPEC>]        Package to run tests for
      --workspace               Test all packages in the workspace
      --exclude <SPEC>          Exclude packages from the test
      --all                     Alias for --workspace (deprecated)
  -j, --jobs <N>                Number of parallel jobs, defaults to # of CPUs
      --keep-going              Do not abort the build as soon as there is an error (unstable)
  -r, --release                 Build artifacts in release mode, with optimizations
      --profile <PROFILE-NAME>  Build artifacts with the specified profile
  -F, --features <FEATURES>     Space or comma separated list of features to activate
      --all-features            Activate all available features
      --no-default-features     Do not activate the `default` feature
      --target <TRIPLE>         Build for the target triple
      --target-dir <DIRECTORY>  Directory for all generated artifacts
      --manifest-path <PATH>    Path to Cargo.toml
      --ignore-rust-version     Ignore `rust-version` specification in packages
      --message-format <FMT>    Error format
      --unit-graph              Output build graph in JSON (unstable)
      --future-incompat-report  Outputs a future incompatibility report at the end of the build
      --timings[=<FMTS>]        Timing output formats (unstable) (comma separated): html, json
  -h, --help                    Print help

Run `cargo help test` for more detailed information.
Run `cargo test -- --help` for test binary options.

should result in something along the lines of

cargo test [OPTIONS] [TESTNAME] [-- [args]...];

<OPTIONS> ::= -q "Display one character per test instead of one line"
        | --quiet "Display one character per test instead of one line"
            | --lib "Test only this package's library unit tests"
            | --bin "Test only the specified binary" [<NAME>]
        [...]
        ;

[Omnikron13]

Running cargo test --help | complgen scrape here definitely doesn't give nice output like that. Don't know if it's a regression, or if it is just incomplete, or what but I get something like this is I try to scrape anything:

 | --doc "Test only this library's documentation"
 | --no-run "Compile, but don't run tests"
 | --no-fail-fast "Run all tests regardless of failure"
 | --ignore-rust-version "Ignore `rust-version` specification in packages"
 | --future-incompat-report "Outputs a future incompatibility report at the end of the build"
 | (--message-format <FMT>) "Error format"
 | (-q | --quiet) "Display one character per test instead of one line"
 | (-v | --verbose...) "Use verbose output (-vv very verbose/build.rs output)"
 | (--color <WHEN>) "Coloring: auto, always, never"
 | (--config <KEY=VALUE>) "Override a configuration value"
 | (-Z <FLAG>) "Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details"
 | (-h | --help) "Print help"
 | (-p [<SPEC>] | --package [<SPEC>]) "Package to run tests for"
 | --workspace "Test all packages in the workspace"
 | (--exclude <SPEC>) "Exclude packages from the test"
 | --all "Alias for --workspace (deprecated)"
 | --lib "Test only this package's library unit tests"
 | --bins "Test all binaries"
 | (--bin [<NAME>]) "Test only the specified binary"
 | --examples "Test all examples"
 | (--example [<NAME>]) "Test only the specified example"
 | --tests "Test all test targets"
 | (--test [<NAME>]) "Test only the specified test target"
 | --benches "Test all bench targets"
 | (--bench [<NAME>]) "Test only the specified bench target"
 | --all-targets "Test all targets (does not include doctests)"
 | (-F <FEATURES> | --features <FEATURES>) "Space or comma separated list of features to activate"
 | --all-features "Activate all available features"
 | --no-default-features "Do not activate the `default` feature"
 | (-j <N> | --jobs <N>) "Number of parallel jobs, defaults to # of CPUs."
 | (-r | --release) "Build artifacts in release mode, with optimizations"
 | (--profile <PROFILE-NAME>) "Build artifacts with the specified profile"
 | (--target [<TRIPLE>]) "Build for the target triple"
 | (--target-dir <DIRECTORY>) "Directory for all generated artifacts"
 | --unit-graph "Output build graph in JSON (unstable)"
 | --timings[=<FMTS>] "Timing output formats (unstable) (comma separated): html, json"
 | (--manifest-path <PATH>) "Path to Cargo.toml"
 | --frozen "Require Cargo.lock and cache are up to date"
 | --locked "Require Cargo.lock is up to date"
 | --offline "Run without accessing the network"

Maybe that is a part of a valid grammar, I don't know, but it definitely seem to be missing the start of the grammar at the least.

[adaszko]

The scrape command is for generating a skeleton of a grammar. It doesn't produce the full grammar and never will.

[Omnikron13]

Doesn't seem to even be including the name of the command as it stands...