search

tag1consulting / goose

Load testing framework, inspired by Locust
https://tag1.com/goose
Apache License 2.0
791 stars 70 forks source link

The `--help` output prints everything twice (kind of) #482

Closed dpc closed 2 years ago

dpc commented 2 years ago

https://docs.rs/crate/gumdrop_derive/latest says:

The help attribute (or a type-level doc comment) can be used to provide some introductory text which will precede option help text in the usage string.

Since GooseConfiguration has such a long doc comment that describes all the options, it is being displayed, just to be followed by all the options generated by the gumpdrop itself automatically. So effectively everything is printed twice.

jeremyandrews commented 2 years ago

Wow, I'm not sure when this started happening, thanks for pointed it out. Including an example of the bug:

     Running `target/release/examples/drupal_memcache -h`
Usage: target/release/examples/drupal_memcache [OPTIONS]

Runtime options available when launching a Goose load test.

Custom defaults can be programmatically set for most of these options using the
`GooseDefaults` structure.

Help is generated for all of these options by passing a `-h` flag to an application
built with the Goose Library. For example, using the following command from within the
Goose source tree to run the included `simple` example:

`cargo run --example simple -- -h`

Goose will generate the following output from the [`GooseConfiguration`] structure:

```text
Usage: target/debug/examples/simple [OPTIONS]

Runtime options available when launching a Goose load test.

Optional arguments:
-h, --help                  Displays this help
-V, --version               Prints version information
-l, --list                  Lists all transactions and exits

-H, --host HOST             Defines host to load test (ie http://10.21.32.33)
-u, --users USERS           Sets concurrent users (default: number of CPUs)
-r, --hatch-rate RATE       Sets per-second user hatch rate (default: 1)
-s, --startup-time TIME     Starts users for up to (30s, 20m, 3h, 1h30m, etc)
-t, --run-time TIME         Stops load test after (30s, 20m, 3h, 1h30m, etc)
-G, --goose-log NAME        Enables Goose log file and sets name
-g, --log-level             Increases Goose log level (-g, -gg, etc)
-q, --quiet                 Decreases Goose verbosity (-q, -qq, etc)
-v, --verbose               Increases Goose verbosity (-v, -vv, etc)

Metrics:
--running-metrics TIME      How often to optionally print running metrics
--no-reset-metrics          Doesn't reset metrics after all users have started
--no-metrics                Doesn't track metrics
--no-transaction-metrics    Doesn't track transaction metrics
--no-print-metrics          Doesn't display metrics at end of load test
--no-error-summary          Doesn't display an error summary
--report-file NAME          Create an html-formatted report
--no-granular-report        Disable granular graphs in report file
-R, --request-log NAME      Sets request log file name
--request-format FORMAT     Sets request log format (csv, json, raw, pretty)
--request-body              Include the request body in the request log
-T, --transaction-log NAME  Sets transaction log file name
--transaction-format FORMAT Sets log format (csv, json, raw, pretty)
-E, --error-log NAME        Sets error log file name
--error-format FORMAT       Sets error log format (csv, json, raw, pretty)
-D, --debug-log NAME        Sets debug log file name
--debug-format FORMAT       Sets debug log format (csv, json, raw, pretty)
--no-debug-body             Do not include the response body in the debug log
--no-status-codes           Do not track status code metrics

Advanced:
--test-plan "TESTPLAN"      Defines a more complex test plan ("10,60s;0,30s")
--no-telnet                 Doesn't enable telnet Controller
--telnet-host HOST          Sets telnet Controller host (default: 0.0.0.0)
--telnet-port PORT          Sets telnet Controller TCP port (default: 5116)
--no-websocket              Doesn't enable WebSocket Controller
--websocket-host HOST       Sets WebSocket Controller host (default: 0.0.0.0)
--websocket-port PORT       Sets WebSocket Controller TCP port (default: 5117)
--no-autostart              Doesn't automatically start load test
--no-gzip                   Doesn't set the gzip Accept-Encoding header
--timeout VALUE             Sets per-request timeout, in seconds (default: 60)
--co-mitigation STRATEGY    Sets coordinated omission mitigation strategy
--throttle-requests VALUE   Sets maximum requests per second
--sticky-follow             Follows base_url redirect with subsequent requests

Gaggle:
--manager                   Enables distributed load test Manager mode
--expect-workers VALUE      Sets number of Workers to expect
--no-hash-check             Tells Manager to ignore load test checksum
--manager-bind-host HOST    Sets host Manager listens on (default: 0.0.0.0)
--manager-bind-port PORT    Sets port Manager listens on (default: 5115)
--worker                    Enables distributed load test Worker mode
--manager-host HOST         Sets host Worker connects to (default: 127.0.0.1)
--manager-port PORT         Sets port Worker connects to (default: 5115)

Goose leverages gumdrop to derive the above help from the the below structure.

Optional arguments: -h, --help Displays this help -V, --version Prints version information -l, --list Lists all transactions and exits

-H, --host HOST Defines host to load test (ie http://10.21.32.33) -u, --users USERS Sets concurrent users (default: number of CPUs) -r, --hatch-rate RATE Sets per-second user hatch rate (default: 1) -s, --startup-time TIME Starts users for up to (30s, 20m, 3h, 1h30m, etc) -t, --run-time TIME Stops load test after (30s, 20m, 3h, 1h30m, etc) -G, --goose-log NAME Enables Goose log file and sets name -g, --log-level Increases Goose log level (-g, -gg, etc) -q, --quiet Decreases Goose verbosity (-q, -qq, etc) -v, --verbose Increases Goose verbosity (-v, -vv, etc)

Metrics: --running-metrics TIME How often to optionally print running metrics --no-reset-metrics Doesn't reset metrics after all users have started --no-metrics Doesn't track metrics --no-transaction-metrics Doesn't track transaction metrics --no-print-metrics Doesn't display metrics at end of load test --no-error-summary Doesn't display an error summary --report-file NAME Create an html-formatted report --no-granular-report Disable granular graphs in report file -R, --request-log NAME Sets request log file name --request-format FORMAT Sets request log format (csv, json, raw, pretty) --request-body Include the request body in the request log -T, --transaction-log NAME Sets transaction log file name --transaction-format FORMAT Sets log format (csv, json, raw, pretty) -E, --error-log NAME Sets error log file name --error-format FORMAT Sets error log format (csv, json, raw, pretty) -D, --debug-log NAME Sets debug log file name --debug-format FORMAT Sets debug log format (csv, json, raw, pretty) --no-debug-body Do not include the response body in the debug log --no-status-codes Do not track status code metrics

Advanced: --test-plan "TESTPLAN" Defines a more complex test plan ("10,60s;0,30s") --no-telnet Doesn't enable telnet Controller --telnet-host HOST Sets telnet Controller host (default: 0.0.0.0) --telnet-port PORT Sets telnet Controller TCP port (default: 5116) --no-websocket Doesn't enable WebSocket Controller --websocket-host HOST Sets WebSocket Controller host (default: 0.0.0.0) --websocket-port PORT Sets WebSocket Controller TCP port (default: 5117) --no-autostart Doesn't automatically start load test --no-gzip Doesn't set the gzip Accept-Encoding header --timeout VALUE Sets per-request timeout, in seconds (default: 60) --co-mitigation STRATEGY Sets coordinated omission mitigation strategy --throttle-requests VALUE Sets maximum requests per second --sticky-follow Follows base_url redirect with subsequent requests

Gaggle: --manager Enables distributed load test Manager mode --expect-workers VALUE Sets number of Workers to expect --no-hash-check Tells Manager to ignore load test checksum --manager-bind-host HOST Sets host Manager listens on (default: 0.0.0.0) --manager-bind-port PORT Sets port Manager listens on (default: 5115) --worker Enables distributed load test Worker mode --manager-host HOST Sets host Worker connects to (default: 127.0.0.1) --manager-port PORT Sets port Worker connects to (default: 5115)