Automatic tests filtering for ComplianceAsCode/content project.
The project analyses changed files, then based on changes decides what has been affected, and at the end selects what should be tested.
Fedora/RHEL:
yum install python3 git
pip3 install gitpython PyYAML deepdiff Jinja2 xmldiff
git clone https://github.com/mildas/content-test-filtering
WARNING: For analysis of Jinja files, ComplianceAsCode dependencies are required. See ComplianceAsCode Developer Guide
How to obtain changes:
positional arguments:
{pr,branch} Subcommands: pr, branch
pr Compare base against selected pull request
branch Compare base against selected branch
Both options have common optional arguments:
-h, --help show this help message and exit
--base BASE_BRANCH Base branch against which we want to compare, default is "master" branch
--repository REPOSITORY_PATH
Path to ComplianceAsCode repository. If not provided, the repository will be cloned into
/tmp folder.
--remote_repo REMOTE_REPO
Remote repository for pulling, updating and finding branches (Pull requests). Default is
https://github.com/ComplianceAsCode/content.
--local Do not pull from remote, use local branches. Mainly for testing purposes.
--verbose Increase verbose level of logging to DEBUG level. Default level is INFO
--output-tests Output only list of tests. Completely turns off all other outputs.
--output-format {raw,markdown}
Output format.
--output {commands,json}
Output from the tool.
--profile Print only profile tests.
--rule Print only rule tests.
By default, changes are obtained from remote repository. When no optional arguments are passed, ComplianceAsCode/content repository is clonned, branch with changes fetched, and then localy analysed.
To disable clonning of the repository and fetch only changes, --repository
is needed.
To perform local changes analysis, branch
(Pull Request numbering is unknown in local repository), --local
(disable branch fetching), and --repository
(path to local repository) arguments must be used.
By default, the project prints human-readable output with identified rule and profile changes and with recommended tests to execute. Github comments support Markdown format, and so to make the output nice for comments, using --output-format markdown
argument outputs it in the format.
For machine-readble output, use --output json
argument. It outputs separate output for profile changes and for rule related changes:
profile
and product
keysrule
, product
, bash
, and ansible
keysTests are located in tests/
directory and are using BATS.
For each script with tests, testing repository is prepared. For each test scenario, setup and clean up phase are done. The setup phase creates temporary file for changes, and clean up puts the repository to previous state.