search

RHSecurityCompliance / contest

Content Testing for ComplianceAsCode/content
Other
4 stars 7 forks source link

Report a `notapplicable` rule if it fails remediation #33

Open comps opened 1 year ago

comps commented 1 year ago

For oscap-based remediations, record somewhere on disk (from the oscap ... --remediate output or from a separate preceding read-only oscap scan) which rules resulted in notapplicable (and some similar other statuses?).

When doing waiving during a final scan, report (in a note) that the rule was originally notapplicable, ie.

FAIL /hardening/oscap/stig/postfix_prevent_unrestricted_relay (notapplicable during remediation)

This makes it possible to easily detect & waive rule dependency and ordering issues.

They shouldn't be auto-waived because a failure might not always be due to ordering - a manual re-run to identify if the rule failed after a double remediation should be still done.

comps commented 1 year ago

This probably slots into https://github.com/RHSecurityCompliance/contest/issues/24 in that we ultimately shouldn't report results in real-time from the output of oscap --verbose, but should instead parse a generated results-arf.xml to get more context for the result (remediation shell output, detailed probe findings, etc.).

In a similar case to https://github.com/RHSecurityCompliance/contest/issues/24, notapplicable-during-remediation can be another piece of metadata from which a final result (for the one rule) is formed and reported after all testing is done.