search

ComplianceAsCode / content-test-filtering

System for Automatic Filtering of Tests for ComplianceAsCode project
8 stars 6 forks source link

Rule.yml analyzer and minor fixes #16

Closed mildas closed 3 years ago

mildas commented 3 years ago

rule.yml analyzer Rule tests selected when:

Product build selected when:

Other cases - nothing selected.

If a rule.yml uses a Jinja macro (if {{{ in rule.yml), then CTF builds content and compares builded rule.yml contents. The main reason for this is that yaml.safe_load is not able to parse some rule.yml files with Jinja macros example. Unfortunately, build of all products takes several minutes.

Minor fixed:

ggbecker commented 3 years ago

rule.yml analyzer Rule tests selected when:

* new rule.yml

* `template:` section in rule.yml changed or added

Product build selected when:

* unknown (a section that is not on list of known sections) section added

Other cases - nothing selected.

If a rule.yml uses a Jinja macro (if {{{ in rule.yml), then CTF builds content and compares builded rule.yml contents. The main reason for this is that yaml.safe_load is not able to parse some rule.yml files with Jinja macros example. Unfortunately, build of all products takes several minutes.

Do we really need to expand those jinja macros? Can't we just find and replace macros? or maybe use dummy definitions. Building the products just for that seems a big overhead.

Minor fixed:

* when a updated rule is not selected in any profile, set its product to `rhel8` - @ggbecker does it make sense?

I think so.

* select only Bash (Ansible) test when a Bash (Ansible) remediation is changed. Otherwise, select both tests (Bash+Ansible)
mildas commented 3 years ago

@ggbecker updated and ready for next review round. I have changed the analysis to the approach where it checks everything after template: and if there's change, then tests are selected. No unknown sections etc. are analyzed, because CTF doesn't handle rule.yml as yaml files anymore.