taylor
commented
2 years ago Estimate of effort:
- @denverwilliams 13
- @agentpoyo 13
- @HashNuke 13
- @taylor 13
- @lixuna 13
- @wavell 13
Closed lixuna closed 2 years ago
taylor
commented
2 years ago Estimate of effort:
agentpoyo
commented
2 years ago Running on a vanilla k3s install.
First issue was running cnf_setup but it still installed coredns pod:
root@cnfqa1:~/v0.28.1# ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
Successfully created directories for cnf-testsuite
cnf-testsuite namespace already exists on the Kubernetes cluster
cnf setup online mode
Successfully setup coredns
/usr/share/crystal/src/regex/match_data.cr:149:7 in 'parse'
/workspace/src/tasks/utils/utils.cr:388:8 in 'version_less_than'
/workspace/src/tasks/utils/cnf_manager.cr:984:8 in 'sample_setup'
/workspace/src/tasks/cnf_setup.cr:37:5 in '->'
/workspace/lib/sam/src/sam/execution.cr:20:7 in '__crystal_main'
/usr/share/crystal/src/crystal/main.cr:110:5 in 'main'
src/env/__libc_start_main.c:94:2 in 'libc_start_main_stage2'
Not a semantic version: "1.22.7k3s1"k3s next error running workload during microservice category which appears to be reasonable_startup_time test:
✔️ PASSED: Image size is good 🐜 ⚖️👀
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: Missing hash key: "data"
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /usr/share/crystal/src/hash.cr:1034:9 in '[]'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /workspace/src/tasks/workload/microservice.cr:209:20 in '->'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /usr/share/crystal/src/primitives.cr:266:3 in 'all_cnfs_task_runner'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /workspace/src/tasks/utils/task.cr:19:9 in 'task_runner'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /workspace/src/tasks/workload/microservice.cr:192:3 in '->'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /workspace/lib/sam/src/sam/task.cr:44:29 in 'call'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /workspace/lib/sam/src/sam/task.cr:44:29 in 'call'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /workspace/lib/sam/src/sam/execution.cr:20:7 in '__crystal_main'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: /usr/share/crystal/src/crystal/main.cr:110:5 in 'main'
E, [2022-04-22 23:15:57 +00:00 #80202] ERROR -- cnf-testsuite: src/env/__libc_start_main.c:94:2 in 'libc_start_main_stage2'
✔️ PASSED: Only one process type used 🐜 ⚖️👀Deferred
agentpoyo
commented
2 years ago agentpoyo
commented
2 years ago Platforms that saw no major errors or issues:
agentpoyo
commented
2 years ago Kind results for comparisons:
pair@cnfdev03:~/workspace/drew/cnf-testsuite$ ./cnf-testsuite cert
Compatibility, Installability & Upgradability Tests
Successfully created directories for cnf-testsuite
✔️ PASSED: Helm Chart exported_chart Lint Passed ⎈📝☑️
✔️ PASSED: Published Helm Chart Found ⎈📦🌐
⏭️ SKIPPED: Helm Deploy
Global docker found. Version: 0.10.1
No Local docker version found
✔️ PASSED: CNF compatible with both Calico and Cilium 🔓🔑
✔️ PASSED: Replicas increased to 3 and decreased to 1 📦📉
Rolling update failed on resource: coredns-coredns and container: coredns
✖️ FAILED: CNF Rollback Failed
Compatibility, installability, and upgradeability results: 115 of 11 tests passed
State Tests
✔️ PASSED: hostPath volumes not found 🖥️ 💾
Rescued: On resource coredns-coredns of kind Service, local storage configuration volumes not found 🖥️ 💾
✔️ PASSED: local storage configuration volumes not found 🖥️ 💾
✔️ FAILED: Volumes used are not elastic volumes 🧫
⏭️ SKIPPED: Mysql not installed 🧫
✔️ 🏆 PASSED: node_drain chaos test passed 🗡️💀♻️
State results: 110 of 5 tests passed
Security Tests
✔️ PASSED: No privileged containers 🔓🔑
⏭️ SKIPPED: Skipping non_root_user: Falco failed to install. Check Kernel Headers are installed on the Host Systems(K8s).
✔️ PASSED: No containers allow a symlink attack 🔓🔑
✖️ FAILED: Found containers that allow privilege escalation 🔓🔑
container :coredns in Deployment: coredns-coredns allow privilege escalation
Remediation: If your application does not need it, make sure the allowPrivilegeEscalation field of the securityContext is set to false.
✔️ PASSED: Containers with insecure capabilities were not found 🔓🔑
✔️ PASSED: Containers with dangerous capabilities were not found 🔓🔑
✖️ 🏆 FAILED: Found containers without resource limits defined 🔓🔑
there are no resource limits defined for container : kubectl-label
Remediation: Define LimitRange and ResourceQuota policies to limit resource usage for namespaces or nodes.
✖️ FAILED: Found resources that do not use security services 🔓🔑
Workload: coredns-coredns does not define any linux security hardening
Remediation: You can use AppArmor, Seccomp, SELinux and Linux Capabilities mechanisms to restrict containers abilities to utilize unwanted privileges.
✖️ FAILED: Ingress and Egress traffic not blocked on pods 🔓🔑
Deployment: coredns-coredns has Pods which don't have ingress/egress defined
Job: gatekeeper-update-namespace-label has Pods which don't have ingress/egress defined
Remediation: Define a network policy that restricts ingress and egress connections.
✔️ PASSED: No containers with hostPID and hostIPC privileges 🔓🔑
✔️ 🏆 PASSED: Containers are running with non-root user with non-root group membership 🔓🔑
✔️ 🏆 PASSED: No privileged containers were found 🔓🔑
✖️ FAILED: Found namespaces which do not have network policies defined 🔓🔑
no policy is defined for namespace default
Remediation: Define network policies or use similar network protection mechanisms.
✖️ FAILED: Found containers with mutable file systems 🔓🔑
container :coredns in Deployment: coredns-coredns has mutable filesystem
Remediation: Set the filesystem of the container to read-only when possible (POD securityContext, readOnlyRootFilesystem: true). If containers application needs to write into the filesystem, it is recommended to mount secondar
y filesystems for specific directories where application require write access.
✔️ PASSED: Containers do not have hostPath mounts 🔓🔑
✔️ 🏆 PASSED: Container engine daemon sockets are not mounted as volumes 🔓🔑
✔️ PASSED: Services are not using external IPs 🔓🔑
⏭️ 🏆 N/A: Pods are not using SELinux 🔓🔑
✔️ PASSED: No restricted values found for sysctls 🔓🔑
✔️ PASSED: No host network attached to pod 🔓🔑
✖️ FAILED: Service accounts automatically mapped 🔓🔑
the following service account: default in the following namespace: default mounts service account tokens in pods by default
the following service account: disk-fill-sa in the following namespace: default mounts service account tokens in pods by default
the following service account: node-drain-sa in the following namespace: default mounts service account tokens in pods by default
the following service account: pod-delete-sa in the following namespace: default mounts service account tokens in pods by default
the following service account: pod-io-stress-sa in the following namespace: default mounts service account tokens in pods by default
the following service account: pod-memory-hog-sa in the following namespace: default mounts service account tokens in pods by default
the following service account: pod-network-corruption-sa in the following namespace: default mounts service account tokens in pods by default
the following service account: pod-network-duplication-sa in the following namespace: default mounts service account tokens in pods by default
the following service account: pod-network-latency-sa in the following namespace: default mounts service account tokens in pods by default
Remediation: Disable automatic mounting of service account tokens to PODs either at the service account level or at the individual POD level, by specifying the automountServiceAccountToken: false. Note that POD level takes pre
cedence.
✔️ PASSED: No applications credentials in configuration files 🔓🔑
Security results: 445 of 21 tests passed
Configuration Tests
✔️ PASSED: No IP addresses found
✔️ PASSED: NodePort is not used
✔️ 🏆 PASSED: HostPort is not used
✔️ 🏆 PASSED: No hard-coded IP addresses found in the runtime K8s configuration
No Secret Volumes or Container secretKeyRefs found for resource: {kind: "Deployment", name: "coredns-coredns", namespace: "default"}
⏭ SKIPPED: Secrets not used 🧫
To address this issue please see the USAGE.md documentation
✖️ FAILED: Found mutable configmap(s) ⚖️
⏭️ SKIPPED: alpha_k8s_apis
✖️ FAILED: Pods should have the app.kubernetes.io/name label. 🏷️❌
Job gatekeeper-update-namespace-label in default namespace failed. validation error: The label `app.kubernetes.io/name` is required. Rule autogen-check-for-labels failed at path /spec/template/metadata/labels/app.kubernetes.io/name/
✔️ 🏆 PASSED: Container images are not using the latest tag 🏷️✔️
✖️ FAILED: Resources are created in the default namespace 🏷️❌
Pod coredns-coredns-6fc69fdfd7-hkql6 in default namespace failed. validation error: Using 'default' namespace is not allowed. Rule validate-namespace failed at path /metadata/namespace/
Job gatekeeper-update-namespace-label in default namespace failed. validation error: Using 'default' namespace is not allowed for pod controllers. Rule validate-podcontroller-namespace failed at path /metadata/namespace/
Deployment coredns-coredns in default namespace failed. validation error: Using 'default' namespace is not allowed for pod controllers. Rule validate-podcontroller-namespace failed at path /metadata/namespace/
✔️ PASSED: Image uses a versioned tag 🏷️✔️
Configuration results: 310 of 11 tests passed
Observability and Diagnostics Tests
✔️ 🏆 PASSED: Resources output logs to stdout and stderr 📶☠️
⏭️ SKIPPED: Prometheus server not found 📶☠️
⏭️ SKIPPED: Prometheus traffic not configured 📶☠️
⏭️ SKIPPED: Fluentd not configured 📶☠️
⏭️ SKIPPED: Jaeger not configured ⎈🚀
Observability and diagnostics results: 100 of 5 tests passed
Microservice Tests
✔️ PASSED: Image size is good 🐜 ⚖️👀
✔️ PASSED: CNF had a reasonable startup time 🚀
✔️ 🏆 PASSED: Only one process type used 🐜 ⚖️👀
✔️ PASSED: Some containers exposed as a service 🐜 ⚖️👀
⏭️ SKIPPED: [shared_database] No MariaDB containers were found
Microservice results: 115 of 5 tests passed
Reliability, Resilience, and Availability Tests
✔️ PASSED: pod_network_latency chaos test passed 🗡️💀♻️
✔️ PASSED: pod_network_corruption chaos test passed 🗡️💀♻️
✔️ PASSED: disk_fill chaos test passed 🗡️💀♻️
✔️ PASSED: pod_delete chaos test passed 🗡️💀♻️
✔️ PASSED: pod_memory_hog chaos test passed 🗡️💀♻️
✔️ PASSED: pod_io_stress chaos test passed 🗡️💀♻️
⏭️ SKIPPED: pod_dns_error docker runtime not found 🗡️💀♻️
✔️ PASSED: pod_network_duplication chaos test passed 🗡️💀♻️
✔️ 🏆 PASSED: Helm liveness probe found ⎈🧫
✔️ 🏆 PASSED: Helm readiness probe found ⎈🧫
Reliability, resilience, and availability results: 235 of 10 tests passed
RESULTS SUMMARY
- 39 of 68 total tests passed
- 13 of 13 essential tests passed
Results have been saved to results/cnf-testsuite-results-20220505-113600-398.ymlPeer Reviews will be in each of the following issues (this EPIC has no AC):
Kubespray https://github.com/cncf/cnf-testsuite/issues/1161 K3s - Issue # https://github.com/cncf/cnf-testsuite/issues/1402 Microk8s - Issue # https://github.com/cncf/cnf-testsuite/issues/1403 AWS - Issue # https://github.com/cncf/cnf-testsuite/issues/1404 Azure - Issue # https://github.com/cncf/cnf-testsuite/issues/1406 Google - Issue # https://github.com/cncf/cnf-testsuite/issues/1405
Is your feature request related to a problem? Please describe.
Describe the solution you'd like
List major platforms for K8s