sonobuoy-serviceaccount makes the cluster_admin platform test fail

[sysarch-repo]

Describe the bug The platform test cluster_admin fails with reference to the sonobuoy facility that is part of the testing framework.

To Reproduce Steps to reproduce the behavior:

1. Install and set up 1.3.3
2. Run the cluster_admin platform test
3. See the failed test referring to sonobuoy-serviceaccount:

   🎬 Testing: [cluster_admin]
   Failed resource: ServiceAccount sonobuoy-serviceaccount
   Remediation: You should apply least privilege principle. Make sure cluster admin permissions are granted only when it is absolutely necessary. Don't use subjects with such high permissions for daily operations.

Expected behavior The testing framework shall not interfere with the AUT/cluster.

Device (please complete the following information):

[ec2-user@ip-10-0-110-88 ~]$ uname -a
Linux ip-10-0-110-88.ec2.internal 5.10.223-212.873.amzn2.x86_64 #1 SMP Wed Aug 7 16:53:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

How will this be tested? aka Acceptance Criteria (optional)

Additional context

[martin-mat]

I don't understand why this should be a bug. The test detects service account with cluster admin permissions so it reports it and fails, as per test description https://github.com/cnti-testcatalog/testsuite/blob/main/docs/TEST_DOCUMENTATION.md#cluster-admin

[sysarch-repo]

@martin-mat Thanks for your comment. The use of Sonobuoy is the decision of the CNF testsuite creator, not the user. Therefore, the tools selected for the execution of tests (Sonobuoy for the k8s_conformance test) should not fail any tests that should solely target the AUT of the user. The user cannot fix the Sonobuoy deployment to make the platform test pass for his/her AUT.

[martin-mat]

I cannot get it reproduced. Can you check and give exact reporoduction steps?

[sysarch-repo]

@martin-mat, it looks like the issue is with an orphaned Sonobuoy serviceaccount resource. With this, you must run the entire platform test group (the k8s_conformance test before the cluster_admin test) to run into the issue. Just running the platform:cluster_admin test in isolation will not be enough for providing evidence.

Isolated cluster_admin test (no issue as there is no Sonobuoy serviceaccount resource deployed yet):

• deploy-aut.sh dns CNF configuration validated 📋 Successfully created directories for cnf-testsuite KUBECONFIG is already set. cnf-testsuite namespace already exists on the Kubernetes cluster ClusterTools installed cnf setup start Waiting for resource availability, timeout for each resource is 1800 seconds  Waiting for resource (1/2): [Deployment] dns-dserver Waiting for resource (2/2): [StatefulSet] dns-drouter All CNF resources are up! Successfully setup dns cnf setup complete AUT dns deployed.
• cnf-testsuite platform:cluster_admin 🎬 Testing: [cluster_admin] ✔️ PASSED: [cluster_admin] No users with cluster-admin RBAC permissions were found 🔓🔑

All platform tests (issue due to orphaned Sonobuoy serviceaccount resource after the k8s_conformance test):

• deploy-aut.sh dns CNF configuration validated 📋 Successfully created directories for cnf-testsuite KUBECONFIG is already set. cnf-testsuite namespace already exists on the Kubernetes cluster ClusterTools installed cnf setup start Waiting for resource availability, timeout for each resource is 1800 seconds  Waiting for resource (1/2): [Deployment] dns-dserver Waiting for resource (2/2): [StatefulSet] dns-drouter All CNF resources are up! Successfully setup dns cnf setup complete AUT dns deployed.
• cnf-testsuite platform Successfully created directories for cnf-testsuite 🎬 Testing: [k8s_conformance] ✔️ PASSED: [k8s_conformance] K8s conformance test has no failures  🎬 Testing: [kube_state_metrics] ⏭️ SKIPPED: [kube_state_metrics] Kube State Metrics not in poc mode 📶☠ 🎬 Testing: [node_exporter] ⏭️ SKIPPED: [node_exporter] node exporter not in poc mode 📶☠ 🎬 Testing: [prometheus_adapter] ⏭️ SKIPPED: [prometheus_adapter] prometheus adapter not in poc mode 📶☠ 🎬 Testing: [metrics_server] ⏭️ SKIPPED: [metrics_server] Metrics server not in poc mode 📶☠ Platform Observability results: 0 of 4 tests passed  🎬 Testing: [worker_reboot_recovery] ⏭️ SKIPPED: [worker_reboot_recovery] Node not in destructive mode  Platform Resilience results: 0 of 1 tests passed  🎬 Testing: [oci_compliant] ✔️ PASSED: [oci_compliant] Your platform is using the following runtimes: [containerd://1.7.11] which are OCI compliant runtimes  Platform Hardware And Scheduling results: 1 of 1 tests passed  🎬 Testing: [control_plane_hardening] ✔️ PASSED: [control_plane_hardening] Insecure port of Kubernetes API server is not enabled 🔓🔑 🎬 Testing: [cluster_admin] Failed resource: ServiceAccount sonobuoy-serviceaccount Remediation: You should apply least privilege principle. Make sure cluster admin permissions are granted only when it is absolutely necessary. Don't use subjects with such high permissions for daily operations. ✖️ FAILED: [cluster_admin] Users with cluster-admin RBAC permissions found 🔓🔑 🎬 Testing: [helm_tiller] ✔️ PASSED: [helm_tiller] No Helm Tiller containers are running 🔓🔑 Platform Security results: 2 of 3 tests passed  Final platform score: 20 of 55 Test results have been saved to results/cnf-testsuite-results-20240919-105528-726.yml