Open ajvb opened 3 years ago
@g-k @sciurus does this capture the various discussions around this well enough or did I miss stuff?
Yep!
Sounds great! Nice summary.
Implementation proposal:
frost glob list <test path glob>
which uses https://docs.python.org/3/library/glob.html to search for and print tests and is roughly equivalent to frost list | grep -i ec2
frost glob test <test path glob>
which runs the tests and is roughly equivalent to frost test $(frost list | grep -i ec2)
frost pytest
as an alias for frost test
and deprecate frost test
to use test-files
or something similar as the default test
command (optional)It'd be better to use pytest markers, but that'd require moving data fetching from the pytest collect phase, which would be a larger change.
@g-k Why not wrap the -m MARKEXPR
pytest CLI argument? But as well, it might be worth defining the groupings we want first.
A feature that has been asked for a couple of times is the ability to have predefined groups of tests that can be run, instead of having to list each test individually within a
frost test ...
command.These could be groups like "security" and "cost-optimization". As well they could be grouped by severity, especially the security related tests.
There seem to be two main reasons for wanting this:
frost test starter --aws-profile my-account
could be a great beginner UX for Frost.