Closed johnsca closed 7 years ago
Rendered --help
:
usage: matrix [-h] [-l LOG_LEVEL] [-L [LOG_NAME [LOG_NAME ...]]]
[-f [LOG_FILTER [LOG_FILTER ...]]] [-s {tui,raw}] [-x FILENAME]
[-i INTERVAL] [-p PATH] [-D] [-B]
[-t [TEST_PATTERN [TEST_PATTERN ...]]] [-g GLITCH_PLAN]
[-n GLITCH_NUM] [-o GLITCH_OUTPUT]
[additional_suites [additional_suites ...]]
Run a local bundle through a suite of tests to ensure that it can handle the
types of operations and failures that are common for all deployments.
positional arguments:
additional_suites Additional suites to be merged with the default suite
before running
optional arguments:
-h, --help show this help message and exit
-l LOG_LEVEL, --log-level LOG_LEVEL
-L [LOG_NAME [LOG_NAME ...]], --log-name [LOG_NAME [LOG_NAME ...]]
-f [LOG_FILTER [LOG_FILTER ...]], --log-filter [LOG_FILTER [LOG_FILTER ...]]
-s {tui,raw}, --skin {tui,raw}
-x FILENAME, --xunit FILENAME
Create an XUnit report file
-i INTERVAL, --interval INTERVAL
-p PATH, --path PATH Path to local bundle to test (defaults to current
directory)
-D, --no-default-suite
Do not include the default suite as the first suite
-B, --no-bundle-suite
Do not include the suite provided by the bundle
-t [TEST_PATTERN [TEST_PATTERN ...]], --test_pattern [TEST_PATTERN [TEST_PATTERN ...]]
Pattern for selecting which tests from the suite(s)
are run
-g GLITCH_PLAN, --glitch_plan GLITCH_PLAN
-n GLITCH_NUM, --glitch_num GLITCH_NUM
-o GLITCH_OUTPUT, --glitch_output GLITCH_OUTPUT
A default suite of tests will always be run, unless -D is given. Additional
suites can be passed in as arguments. Additionally, the bundle will be
checked for a tests/matrix.yaml file which will be included as an additional
suite if present. All suites will be merged together, and suites containing
tests with the same name as a test in a previous suite will override that
test.
Examples:
Run all normal tests (default suite and, if present, ./tests/matrix.yaml)
on the bundle at ./:
$ matrix
Run only the default suite (ignore ./tests/matrix.yaml):
$ matrix -B
Run only ~/foo/tests/matrix.yaml on the bundle at ~/foo:
$ matrix -Dp ~/foo
Run all normal tests plus ./tests/matrix_extra.yaml on the bundle at ./:
$ matrix tests/matrix_extra.yaml
Run only ./tests/matrix_extra.yaml:
$ matrix -DB tests/matrix_extra.yaml
LGTM/+1
Currently unable to test due to juju-solutions/python-libjuju#7 and https://bugs.launchpad.net/juju/+bug/1645917