search

openshift / compliance-operator

Operator providing OpenShift cluster compliance checks
Apache License 2.0
110 stars 110 forks source link

inconsistent compliancecheckresults due to "notapplicable" checks on nodes #834

Closed felixkrohn closed 2 years ago

felixkrohn commented 2 years ago

Hi there, On an ocp4.10 Cluster (IPI on AWS -> using rhcos) I see some inconsistencies in compliancecheckresults since today. It seems to boil down to some checks considered "notapplicable" on 2 Nodes (out of 44). Both of these nodes are older than a month with the cluster itself being approximately 2y96d old. Upon examination, the following checks report inconsistent status: [1] When diffing 2 of the resulting Configmaps with the OSCAP Scan Results (see diff [2]), I see that the instance reporting issues has 2 missing "platform idref" lines in comparison to one where the checks are executed. What could be the cause for this, how to dig deeper into this?

[1] 

$ oc get compliancecheckresults -l compliance.openshift.io/inconsistent-check -oname
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-kernel-module-loading-delete
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-kernel-module-loading-finit
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-kernel-module-loading-init
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-login-events-lastlog
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-login-events-tallylog
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-mac-modification
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-networkconfig-modification
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-session-events
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-sysadmin-actions
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-time-adjtimex
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-time-settimeofday
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-time-watch-localtime
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-usergroup-modification-group
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-audit-rules-usergroup-modification-passwd
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-data-retention-flush
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-data-retention-max-log-file
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-data-retention-max-log-file-action
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-data-retention-num-logs
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-data-retention-space-left
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-data-retention-space-left-action
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-freq
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-local-events
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-log-format
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-name-format
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-auditd-write-logs
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-bios-disable-usb-boot
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-chronyd-client-only
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-chronyd-no-chronyc-network
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-chronyd-or-ntpd-specify-multiple-servers
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-coreos-audit-backlog-limit-kernel-argument
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-coreos-audit-option
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-coreos-enable-selinux-kernel-argument
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-coreos-page-poison-kernel-argument
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-directory-access-var-log-audit
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-directory-permissions-var-log-audit
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-disable-ctrlaltdel-reboot
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-file-groupowner-sshd-config
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-file-owner-sshd-config
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-file-ownership-var-log-audit
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-file-permissions-sshd-config
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-file-permissions-sshd-private-key
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-file-permissions-sshd-pub-key
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-atm-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-bluetooth-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-can-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-cfg80211-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-cramfs-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-firewire-core-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-freevxfs-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-hfs-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-hfsplus-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-iwlmvm-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-iwlwifi-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-jffs2-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-mac80211-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-sctp-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-squashfs-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-tipc-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-udf-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-kernel-module-usb-storage-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-no-tmux-in-shells
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-package-audit-installed
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-package-sudo-installed
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-partition-for-var-log
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-partition-for-var-log-audit
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-require-singleuser-auth
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-selinux-confinement-of-daemons
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-selinux-policytype
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-selinux-state
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-service-auditd-enabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-service-autofs-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-service-bluetooth-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-service-chronyd-or-ntpd-enabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-service-debug-shell-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sshd-limit-user-access
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-fs-protected-hardlinks
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-fs-protected-symlinks
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-kernel-dmesg-restrict
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-kernel-kexec-load-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-kernel-kptr-restrict
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-kernel-perf-event-paranoid
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-kernel-unprivileged-bpf-disabled
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-core-bpf-jit-harden
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-all-accept-redirects
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-all-accept-source-route
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-all-log-martians
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-all-rp-filter
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-all-secure-redirects
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-all-send-redirects
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-default-accept-redirects
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-default-accept-source-route
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-default-log-martians
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-default-rp-filter
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-default-secure-redirects
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-conf-default-send-redirects
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-icmp-echo-ignore-broadcasts
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-icmp-ignore-bogus-error-responses
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv4-tcp-syncookies
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv6-conf-all-accept-ra
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv6-conf-all-accept-redirects
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv6-conf-all-accept-source-route
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv6-conf-default-accept-ra
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv6-conf-default-accept-redirects
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-sysctl-net-ipv6-conf-default-accept-source-route
compliancecheckresult.compliance.openshift.io/corpname-rhcos4-moderate-worker-wireless-disable-in-bios

[2] relevant diff, without timestamp diffs

@@ -7,18 +7,20 @@
               <benchmark href="#scap_org.open-scap_comp_ssg-rhcos4-xccdf.xml" id="xccdf_org.ssgproject.content_benchmark_RHCOS-4"/>
               <title>OSCAP Scan Result</title>
               <profile idref="xccdf_compliance.openshift.io_profile_corpname-rhcos4-moderate"/>
-              <target>ip-10-176-50-19.eu-central-1.compute.internal</target>
+              <target>ip-10-176-50-10.eu-central-1.compute.internal</target>
               <target-facts>
                 <fact name="urn:xccdf:fact:scanner:name" type="string">OpenSCAP</fact>
                 <fact name="urn:xccdf:fact:scanner:version" type="string">1.3.5</fact>
-                <fact name="urn:xccdf:fact:identifier" type="string">ip-10-176-50-19.eu-central-1.compute.internal</fact>
-                <fact name="urn:xccdf:fact:asset:identifier:ein" type="string">ip-10-176-50-19.eu-central-1.compute.internal</fact>
+                <fact name="urn:xccdf:fact:identifier" type="string">ip-10-176-50-10.eu-central-1.compute.internal</fact>
+                <fact name="urn:xccdf:fact:asset:identifier:ein" type="string">ip-10-176-50-10.eu-central-1.compute.internal</fact>
               </target-facts>
               <target-id-ref system="http://scap.nist.gov/schema/asset-identification/1.1" name="asset0" href=""/>
               <platform idref="#grub2"/>
+              <platform idref="#audit"/>
               <platform idref="#not_s390x_arch"/>
               <platform idref="cpe:/o:redhat:enterprise_linux_coreos:4"/>
               <platform idref="#pam"/>
+              <platform idref="#machine"/>
               <platform idref="#systemd"/>
               <set-value idref="xccdf_org.ssgproject.content_value_var_aide_scan_notification_email">root@localhost</set-value>
               <set-value idref="xccdf_org.ssgproject.content_value_var_ssh_client_rekey_limit_size">512M</set-value>
@@ -584,12 +586,14 @@
                 <result>notselected</result>
               </rule-result>
               <rule-result idref="xccdf_org.ssgproject.content_rule_partition_for_var_log" role="full" time="2022-11-08T12:00:00+00:00" severity="low" weight="1.000000">
-                <result>notapplicable</result>
+                <result>notchecked</result>
                 <ident system="https://nvd.nist.gov/cce/index.cfm">CCE-82737-8</ident>
+                <message severity="info">No candidate or applicable check found.</message>
               </rule-result>
               <rule-result idref="xccdf_org.ssgproject.content_rule_partition_for_var_log_audit" role="full" time="2022-11-08T12:00:00+00:00" severity="low" weight="1.000000">
-                <result>notapplicable</result>
+                <result>notchecked</result>
                 <ident system="https://nvd.nist.gov/cce/index.cfm">CCE-82738-6</ident>
+                <message severity="info">No candidate or applicable check found.</message>
               </rule-result>
               <rule-result idref="xccdf_org.ssgproject.content_rule_partition_for_var_tmp" role="full" time="2022-11-08T12:00:00+00:00" severity="medium" weight="1.000000">
                 <result>notselected</result>
[...] similar diff for all other notapplicable checks as well [...]
felixkrohn commented 2 years ago

looks like I have to update my bookmarks... closing here after having opened https://github.com/ComplianceAsCode/compliance-operator/issues/178.