audit_rules_login_events rule is failing several tests

cortesana commented 1 year ago

Description of problem:

The audit_rules_login_events rule is failing in several tests during generation fix for PCI-DSS profile.

/CoreOS/scap-security-guide/hardening/host-os/oscap/pci-dss
/CoreOS/scap-security-guide/hardening/oscap/pci-dss

It is failing for both Ansible and Bash remediations in the PCI-DSS profile.

/CoreOS/scap-security-guide/Sanity/test-rules-scenarios-per-profile
/CoreOS/scap-security-guide/Sanity/machine-hardening PCI-DSS

SCAP Security Guide Version:

Current upstream master branch as of 2023-09-04

Operating System Version:

RHEL 7

Steps to Reproduce:

oscap xccdf generate --profile (all) fix /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml or
python3 /tmp/tmp.tH0laEUiO4/rpmbuild/BUILD/scap-security-guide-0.1.70/tests/test_suite.py combined --slice 2 5 --libvirt qemu:///system test_suite_vm --datastream /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml --mode online --remediate-using ansible --duplicate-templates --no-reports xccdf_org.ssgproject.content_profile_pci-dss

Actual Results:

xccdf_org.ssgproject.content_rule_audit_rules_login_events:fail
lib.oscap._rules_without_remediation:14: running: oscap xccdf generate --profile (all) fix /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml
W: oscap:     Obtrusive data from probe!
W: oscap:     Obtrusive data from probe!
W: oscap:     Obtrusive data from probe!
W: oscap:     Obtrusive data from probe!
W: oscap:     Obtrusive data from probe!
W: oscap:     Obtrusive data from probe!
lib.results.report_plain:171: FAIL audit_rules_login_events

Expected Results:

xccdf_org.ssgproject.content_rule_audit_rules_login_events:pass
test.py:45: lib.results.report_plain:171: PASS audit_rule_audit_rules_login_events

Additional Information/Debugging Steps:

jan-cerny commented 1 year ago

Moreover, the test scenarios for this rule are failing locally:

[jcerny@fedora scap-security-guide{master}]$ python3 tests/automatus.py rule --libvirt qemu:///system ssgts_rhel7 audit_rules_login_events
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/jcerny/work/git/scap-security-guide/logs/rule-custom-2023-09-11-1135/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_audit_rules_login_events
INFO - Script empty.fail.sh using profile (all) OK
ERROR - Rule evaluation resulted in error, instead of expected fixed during remediation stage 
ERROR - The remediation failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_login_events'.
ERROR - Script rules_without_keys.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_login_events'.
ERROR - Script default.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_login_events'.
[jcerny@fedora scap-security-guide{master}]$ python3 tests/automatus.py rule --libvirt qemu:///system ssgts_rhel7 --remediate-using ansible audit_rules_login_events
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/jcerny/work/git/scap-security-guide/logs/rule-custom-2023-09-11-1138/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_audit_rules_login_events
INFO - Script empty.fail.sh using profile (all) OK
ERROR - Script rules_without_keys.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_login_events'.
ERROR - Script default.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_login_events'.

jan-cerny commented 1 year ago

The issue is still present in the latest productization run with scap-security-guide from the latest upstream master as of HEAD 7c741f2 as of 2023-09-09.

ComplianceAsCode / content