Scan our YAML config files with kubescape

sgibson91 commented 2 years ago

Background and proposal

Context kubescape is an open-source tool to perform security scans on k8s resources. Repo: https://github.com/armosec/kubescape It generates security reports using common security frameworks, including the NSA/CISA k8s hardening guidance which is one of the gold-standards in this area. (I recall it was recommended to us during our meeting with CUIT.)

Proposed solution We should implement a scan function in the deployer to scan our YAML config files using kubescape (you can pass the output of helm template directly to kubescape) and run this function in our validation CI/CD (it's probably too verbose to also do it on deploy).

Value and who benefits:

The engineering team will become more aware of what risks there are in our infrastructure and where so we can take steps to address them
It may be useful in leads/partnerships to say something like "similar hubs to the type you'd like have a risk score of X against the Y framework", particularly when dealing with university IT departments who are responsible for security.

Implementation guide and constraints

We should implement a scan function the the deployer to scan our YAML files with kubescape (example in comments)
There is also the capability to scan the running cluster itself and submit results to a cloud service. We should NOT implement this until 1) we understand more about what that entails, and 2) we decide if it's useful for the team.

Updates and ongoing work

No response

sgibson91 commented 2 years ago

Example command running kubescape on the YAML config describing the staging hub on the 2i2c cluster

Command executed:

helm template staging ./helm-charts/basehub -f config/clusters/2i2c/staging.values.yaml --dry-run | kubescape scan -

Output:

[control: Access Kubernetes dashboard - https://hub.armo.cloud/docs/c-0014] passed 👍
Description: Attackers who gain access to the dashboard service account or have its RBAC permissions can use its network access to retrieve information about resources in the cluster or change them. This control checks if a subject that is not dashboard service account is bound to dashboard role/clusterrole, or - if anyone that is not the dashboard pod is associated with dashboard service account.
Summary - Passed:2   Excluded:0   Failed:0   Total:2

[control: Access container service account - https://hub.armo.cloud/docs/c-0053] failed 😥
Description: Attackers who obtain access to a pod can use its SA token to communicate with KubeAPI server. All PODs with SA token mounted (if such token has a Role or a ClusterRole binding) are considerred potentially dangerous.
Failed:
  ClusterRole - staging-user-scheduler 
  ClusterRoleBinding - staging-user-scheduler 
  Deployment - hub 
  Deployment - user-scheduler 
  Role - hub 
  RoleBinding - hub 
Summary - Passed:7   Excluded:0   Failed:6   Total:13
Remediation: Verify that RBAC is enabled. Follow the least privilege principle and ensure that only necessary PODs have SA token mounted into them.

[control: Access tiller endpoint - https://hub.armo.cloud/docs/c-0033] skipped 😕
Description: Attackers may run code on any container that is accessible to the tiller’s service and perform actions in the cluster, using the tiller’s service account, which often has high privileges. This control checks if unauthenticated version of the Tiller runs in the cluster.
[control: Allow privilege escalation - https://hub.armo.cloud/docs/c-0016] failed 😥
Description: Attackers may gain access to a container and uplift its privilege to enable excessive capabilities.
Failed:
  Deployment - docs-service 
  Job - nfs-share-creator 
Summary - Passed:4   Excluded:0   Failed:2   Total:6
Remediation: If your application does not need it, make sure the allowPrivilegeEscalation field of the securityContext is set to false.

[control: Allowed hostPath - https://hub.armo.cloud/docs/c-0006] passed 👍
Description: Mounting host directory to the container can be abused to get access to sensitive data and gain persistence on the host machine.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: Applications credentials in configuration files - https://hub.armo.cloud/docs/c-0012] failed 😥
Description: Attackers who have access to configuration files can steal the stored secrets and use them. This control checks if ConfigMaps or pod specifications have sensitive information in their configuration.
Failed:
  ConfigMap - hub 
Summary - Passed:8   Excluded:0   Failed:1   Total:9
Remediation: Use Kubernetes secrets or Key Management Systems to store credentials.

[control: Audit logs enabled - https://hub.armo.cloud/docs/c-0067] skipped 😕
Description: Audit logging is an important security feature in Kubernetes, it enables the operator to track requests to the cluster. It is important to use it so the operator has a record of events happened in Kubernetes
[control: Automatic mapping of service account - https://hub.armo.cloud/docs/c-0034] passed 👍
Description: Potential attacker may gain access to a POD and steal its service account token. Therefore, it is recommended to disable automatic mapping of the service account tokens in service account configuration and enable it only for PODs that need to use them.
Summary - Passed:9   Excluded:0   Failed:0   Total:9

[control: CVE-2021-25741 - Using symlink for arbitrary host file system access. - https://hub.armo.cloud/docs/c-0058] skipped 😕
Description: A user may be able to create a container with subPath or subPathExpr volume mounts to access files & directories anywhere on the host filesystem. Following Kubernetes versions are affected: v1.22.0 - v1.22.1, v1.21.0 - v1.21.4, v1.20.0 - v1.20.10, version v1.19.14 and lower. This control checks the vulnerable versions and the actual usage of the subPath feature in all Pods in the cluster. If you want to learn more about the CVE, please refer to the CVE link: https://nvd.nist.gov/vuln/detail/CVE-2021-25741
[control: CVE-2021-25742-nginx-ingress-snippet-annotation-vulnerability - https://hub.armo.cloud/docs/c-0059] skipped 😕
Description: Security issue in ingress-nginx where a user that can create or update ingress objects can use the custom snippets feature to obtain all secrets in the cluster (see more at https://github.com/kubernetes/ingress-nginx/issues/7837)
[control: CVE-2022-0185-linux-kernel-container-escape - https://hub.armo.cloud/docs/c-0079] skipped 😕
Description: CVE-2022-0185 is a kernel vulnerability enabling privilege escalation and it can lead attackers to escape containers and take control over nodes. This control alerts on vulnerable kernel versions of Kubernetes nodes
[control: CVE-2022-0492-cgroups-container-escape - https://hub.armo.cloud/docs/c-0086] failed 😥
Description: Linux Kernel vulnerability CVE-2022-0492 may allow malicious code running inside container to escape container isolation and gain root privileges on the entire node. When fixed Kernel version numbers will become available, this control will be modified to verify them and avoid false positive detections. This control identifies all the resources that don't deploy neither AppArmor nor SELinux, run as root or allow privige escalation or have corresponding dangerous capabilities.
Failed:
  Deployment - docs-service 
  Job - nfs-share-creator 
Summary - Passed:4   Excluded:0   Failed:2   Total:6
Remediation: Activate AppArmor or SELinux. Follow the least privilege principle and remove root privileges or privilege escalation option and CAP_DAC_OVERRIDE capability. Make sure you don't allow container images from potentially dangerous sources and that containers that must have high privileges are taken from protected repositories.

[control: CVE-2022-24348-argocddirtraversal - https://hub.armo.cloud/docs/c-0081] skipped 😕
Description: CVE-2022-24348 is a major software supply chain 0-day vulnerability in the popular open source CD platform Argo CD which can lead to privilege escalation and information disclosure.
[control: Cluster internal networking - https://hub.armo.cloud/docs/c-0054] skipped 😕
Description: If no network policy is defined, attackers who gain access to a container may use it to move laterally in the cluster. This control lists namespaces in which no network policy is defined.
[control: Cluster-admin binding - https://hub.armo.cloud/docs/c-0035] passed 👍
Description: Attackers who have cluster admin permissions (can perform any action on any resource), can take advantage of their privileges for malicious activities. This control determines which subjects have cluster admin permissions.
Summary - Passed:4   Excluded:0   Failed:0   Total:4

[control: Configured liveness probe - https://hub.armo.cloud/docs/c-0056] failed 😥
Description: Liveness probe is intended to ensure that workload remains healthy during its entire execution lifecycle, or otherwise restrat the container. It is highly recommended to define liveness probe for every worker container. This control finds all the PODs where the Liveness probe is not configured.
Failed:
  Deployment - docs-service 
  StatefulSet - user-placeholder 
Summary - Passed:3   Excluded:0   Failed:2   Total:5
Remediation: Ensure Liveness probes are configured wherever possible.

[control: Configured readiness probe - https://hub.armo.cloud/docs/c-0018] failed 😥
Description: Readiness probe is intended to ensure that workload is ready to process network traffic. It is highly recommended to define readiness probe for every worker container. This control finds all the PODs where the readiness probe is not configured.
Failed:
  Deployment - docs-service 
  StatefulSet - user-placeholder 
Summary - Passed:3   Excluded:0   Failed:2   Total:5
Remediation: Ensure Readiness probes are configured wherever possible.

[control: Container hostPort - https://hub.armo.cloud/docs/c-0044] passed 👍
Description: Configuring hostPort requires a particular port number. If two objects specify the same HostPort, they could not be deployed to the same node. It may prevent the second object from starting, even if Kubernetes will try reschedule it on another node, provided there are available nodes with sufficient amount of resources. Also, if the number of replicas of such workload is higher than the number of nodes, the deployment will consistently fail.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: Containers mounting Docker socket - https://hub.armo.cloud/docs/c-0074] passed 👍
Description: Mounting Docker socket (Unix socket) enables container to access Docker internals, retrieve sensitive information and execute Docker commands, if Docker runtime is available. This control identifies PODs that attempt to mount Docker socket for accessing Docker runtime.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: Control plane hardening - https://hub.armo.cloud/docs/c-0005] skipped 😕
Description: Kubernetes control plane API is running with non-secure port enabled which allows attackers to gain unprotected access to the cluster.
[control: CoreDNS poisoning - https://hub.armo.cloud/docs/c-0037] passed 👍
Description: If attackers have permissions to modify the coredns ConfigMap they can change the behavior of the cluster’s DNS, poison it, and override the network identity of other services. This control identifies all subjects allowed to update the 'coredns' configmap.
Summary - Passed:7   Excluded:0   Failed:0   Total:7

[control: Data Destruction - https://hub.armo.cloud/docs/c-0007] failed 😥
Description: Attackers may attempt to destroy data and resources in the cluster. This includes deleting deployments, configurations, storage, and compute resources. This control identifies all subjects that can delete resources.
Failed:
  ClusterRole - staging-user-scheduler 
  ClusterRoleBinding - staging-user-scheduler 
  Role - hub 
  RoleBinding - hub 
Summary - Passed:0   Excluded:0   Failed:5   Total:5
Remediation: You should follow the least privilege principle and minimize the number of subjects that can delete resources.

[control: Delete Kubernetes events - https://hub.armo.cloud/docs/c-0031] passed 👍
Description: Attackers may delete Kubernetes events to avoid detection of their activity in the cluster. This control identifies all the subjects that can delete Kubernetes events.
Summary - Passed:4   Excluded:0   Failed:0   Total:4

[control: Disable anonymous access to Kubelet service - https://hub.armo.cloud/docs/c-0069] skipped 😕
Description: By default, requests to the kubelet's HTTPS endpoint that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of system:anonymous and a group of system:unauthenticated.
[control: Enforce Kubelet client TLS authentication - https://hub.armo.cloud/docs/c-0070] skipped 😕
Description: Kubelets are the node level orchestrator in Kubernetes control plane. They are publishing service port 10250 where they accept commands from API server. Operator must make sure that only API server is allowed to submit commands to Kubelet. This is done through client certificate verification, must configure Kubelet with client CA file to use for this purpose.
[control: Exec into container - https://hub.armo.cloud/docs/c-0002] passed 👍
Description: Attackers with relevant permissions can run malicious commands in the context of legitimate containers in the cluster using “kubectl exec” command. This control determines which subjects have permissions to use this command.
Summary - Passed:4   Excluded:0   Failed:0   Total:4

[control: Exposed dashboard - https://hub.armo.cloud/docs/c-0047] passed 👍
Description: Kubernetes dashboard versions before v2.0.1 do not support user authentication. If exposed externally, it will allow unauthenticated remote management of the cluster. This control checks presence of the kubernetes-dashboard deployment and its version number.
Summary - Passed:9   Excluded:0   Failed:0   Total:9

[control: Exposed sensitive interfaces - https://hub.armo.cloud/docs/c-0021] passed 👍
Description: Exposing a sensitive interface to the internet poses a security risk. It might enable attackers to run malicious code or deploy containers in the cluster. This control checks if known components (e.g. Kubeflow, Argo Workflows, etc.) are deployed and exposed services externally.
Summary - Passed:11   Excluded:0   Failed:0   Total:11

[control: Forbidden Container Registries - https://hub.armo.cloud/docs/c-0001] failed 😥
Description: In cases where the Kubernetes cluster is provided by a CSP (e.g., AKS in Azure, GKE in GCP, or EKS in AWS), compromised cloud credential can lead to the cluster takeover. Attackers may abuse cloud account credentials or IAM mechanism to the cluster’s management layer.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Job - nfs-share-creator 
Summary - Passed:3   Excluded:0   Failed:3   Total:6
Remediation: Limit the registries from which you pull container images from

[control: Host PID/IPC privileges - https://hub.armo.cloud/docs/c-0038] passed 👍
Description: Containers should be isolated from the host machine as much as possible. The hostPID and hostIPC fields in deployment yaml may allow cross-container influence and may expose the host itself to potentially malicious or destructive actions. This control identifies all PODs using hostPID or hostIPC privileges.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: HostNetwork access - https://hub.armo.cloud/docs/c-0041] passed 👍
Description: Potential attackers may gain access to a POD and inherit access to the entire host network. For example, in AWS case, they will have access to the entire VPC. This control identifies all the PODs with host network access enabled.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: HostPath mount - https://hub.armo.cloud/docs/c-0048] passed 👍
Description: Mounting host directory to the container can be used by attackers to get access to the underlying host. This control identifies all the PODs using hostPath mount.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: Image pull policy on latest tag - https://hub.armo.cloud/docs/c-0075] passed 👍
Description: While usage of the latest tag is not generally recommended, in some cases this is necessary. If it is, the ImagePullPolicy must be set to Always, otherwise Kubernetes may run an older image with the same name that happens to be present in the node cache. Note that using Always will not cause additional image downloads because Kubernetes will check the image hash of the local local against the registry and only pull the image if this hash has changed, which is exactly what users want when use the latest tag. This control will identify all PODs with latest tag that have ImagePullSecret not set to Always.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: Images from allowed registry - https://hub.armo.cloud/docs/c-0078] failed 😥
Description: This control is intended to ensure that all the used container images are taken from the authorized repositories. It allows user to list all the approved repositories and will fail all the images taken from any repository outside of this list.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Deployment - proxy 
  Job - nfs-share-creator 
Summary - Passed:2   Excluded:0   Failed:4   Total:6
Remediation: You should enable all trusted repositories in the parameters of this control.

[control: Immutable container filesystem - https://hub.armo.cloud/docs/c-0017] failed 😥
Description: Mutable container filesystem can be abused to inject malicious code or data into containers. Use immutable (read-only) filesystem to limit potential attacks.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Deployment - proxy 
  Deployment - user-scheduler 
  Job - nfs-share-creator 
  StatefulSet - user-placeholder 
Summary - Passed:0   Excluded:0   Failed:6   Total:6
Remediation: Set the filesystem of the container to read-only when possible (POD securityContext, readOnlyRootFilesystem: true). If containers application needs to write into the filesystem, it is recommended to mount secondary filesystems for specific directories where application require write access.

[control: Ingress and Egress blocked - https://hub.armo.cloud/docs/c-0030] failed 😥
Description: Disable Ingress and Egress traffic on all pods wherever possible. It is recommended to define restrictive network policy on all new PODs, and then enable sources/destinations that this POD must communicate with.
Failed:
  Deployment - docs-service 
  Deployment - user-scheduler 
  Job - nfs-share-creator 
  StatefulSet - user-placeholder 
Summary - Passed:5   Excluded:0   Failed:4   Total:9
Remediation: Define a network policy that restricts ingress and egress connections.

[control: Insecure capabilities - https://hub.armo.cloud/docs/c-0046] passed 👍
Description: Giving insecure or excsessive capabilities to a container can increase the impact of the container compromise. This control identifies all the PODs with dangerous capabilities (see documentation pages for details).
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: K8s common labels usage - https://hub.armo.cloud/docs/c-0077] failed 😥
Description: Kubernetes common labels help manage and monitor Kubernetes cluster using different tools such as kubectl, dashboard and others in an interoperable way. Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/ for more information. This control helps you find objects that don't have any of these labels defined.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Deployment - proxy 
  Deployment - user-scheduler 
  Job - nfs-share-creator 
  StatefulSet - user-placeholder 
Summary - Passed:0   Excluded:0   Failed:6   Total:6
Remediation: Define applicable labels or use the exception mechanism to prevent further notifications.

[control: Kubernetes CronJob - https://hub.armo.cloud/docs/c-0026] skipped 😕
Description: Attackers may use Kubernetes CronJob for scheduling execution of malicious code that would run as a POD in the cluster. This control lists all the CronJobs that exist in the cluster for the user to approve.
[control: Label usage for resources - https://hub.armo.cloud/docs/c-0076] failed 😥
Description: It is recommended to set labels that identify semantic attributes of your application or deployment. For example, { app: myapp, tier: frontend, phase: test, deployment: v3 }. These labels can used to assign policies to logical groups of the deployments as well as for presentation and tracking purposes. This control helps you find deployments without any of the expected labels.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Deployment - proxy 
  Deployment - user-scheduler 
  Job - nfs-share-creator 
  StatefulSet - user-placeholder 
Summary - Passed:0   Excluded:0   Failed:6   Total:6
Remediation: Define labels that are most suitable to your needs of use the exceptions to prevent further notifications.

[control: Linux hardening - https://hub.armo.cloud/docs/c-0055] failed 😥
Description: Containers may be given more privileges than they actually need. This can increase the potential impact of a container compromise.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Deployment - proxy 
  Deployment - user-scheduler 
  Job - nfs-share-creator 
  StatefulSet - user-placeholder 
Summary - Passed:0   Excluded:0   Failed:6   Total:6
Remediation: You can use AppArmor, Seccomp, SELinux and Linux Capabilities mechanisms to restrict containers abilities to utilize unwanted privileges.

[control: List Kubernetes secrets - https://hub.armo.cloud/docs/c-0015] failed 😥
Description: Attackers who have permissions to access secrets can access sensitive information that might include credentials to various services. This control determines which user, group or service account can list/get secrets.
Failed:
  Role - hub 
  RoleBinding - hub 
Summary - Passed:2   Excluded:0   Failed:3   Total:5
Remediation: Monitor and approve list of users, groups and service accounts that can access secrets. Use exception mechanism to prevent repetitive the notifications.

[control: Malicious admission controller (mutating) - https://hub.armo.cloud/docs/c-0039] skipped 😕
Description: Attackers may use mutating webhooks to intercept and modify all the resources in the cluster. This control lists all mutating webhook configurations that must be verified.
[control: Malicious admission controller (validating) - https://hub.armo.cloud/docs/c-0036] skipped 😕
Description: Attackers can use validating webhooks to intercept and discover all the resources in the cluster. This control lists all the validating webhook configurations that must be verified.
[control: Mount service principal - https://hub.armo.cloud/docs/c-0020] passed 👍
Description: When a cluster is deployed in the cloud, in some cases attackers can leverage their access to a container in the cluster to gain cloud credentials. This control determines if any workload contains a hostPath volume.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: Naked PODs - https://hub.armo.cloud/docs/c-0073] skipped 😕
Description: It is not recommended to create PODs without parental Deployment, ReplicaSet, StatefulSet etc.Manual creation if PODs may lead to a configuration drifts and other untracked changes in the system. Such PODs won't be automatically rescheduled by Kubernetes in case of a crash or infrastructure failure. This control identifies every POD that does not have corresponding parental object.
[control: Namespace without service accounts - https://hub.armo.cloud/docs/c-0060] passed 👍
Description: It is recommended not to use default service account anywhere in production environment. This control identifies all namespaces without explicit non-default service account.
Summary - Passed:3   Excluded:0   Failed:0   Total:3

[control: Network mapping - https://hub.armo.cloud/docs/c-0049] skipped 😕
Description: If no network policy is defined, attackers who gain access to a single container may use it to probe the network. This control lists all namespaces in which no network policies are defined.
[control: No impersonation - https://hub.armo.cloud/docs/c-0065] passed 👍
Description: Impersonation is an explicit RBAC permission to use other roles rather than the one assigned to a user, group or service account. This is sometimes needed for testing purposes. However, it is highly recommended not to use this capability in the production environments for daily operations. This control identifies all subjects whose roles include impersonate verb.
Summary - Passed:4   Excluded:0   Failed:0   Total:4

[control: Non-root containers - https://hub.armo.cloud/docs/c-0013] failed 😥
Description: Potential attackers may gain access to a container and leverage its existing privileges to conduct an attack. Therefore, it is not recommended to deploy containers with root privileges unless it is absolutely necessary. This control identifies all the Pods running as root or can escalate to root.
Failed:
  Deployment - docs-service 
  Job - nfs-share-creator 
Summary - Passed:4   Excluded:0   Failed:2   Total:6
Remediation: If your application does not need root privileges, make sure to define the runAsUser or runAsGroup under the PodSecurityContext and use user ID 1000 or higher. Do not turn on allowPrivlegeEscalation bit and make sure runAsNonRoot is true.

[control: PSP enabled - https://hub.armo.cloud/docs/c-0068] skipped 😕
Description: PSP enable fine-grained authorization of pod creation and it is important to enable it
[control: Pods in default namespace - https://hub.armo.cloud/docs/c-0061] failed 😥
Description: It is recommended to avoid running PODs in cluster without explicit namespace assignment. This control identifies all the PODs running in the default namespace.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Deployment - proxy 
  Deployment - user-scheduler 
  Job - nfs-share-creator 
  StatefulSet - user-placeholder 
Summary - Passed:0   Excluded:0   Failed:6   Total:6
Remediation: Create necessary namespaces and move all the PODs from default namespace there.

[control: Portforwarding privileges - https://hub.armo.cloud/docs/c-0063] passed 👍
Description: Attackers with relevant RBAC permission can use “kubectl portforward” command to establish direct communication with PODs from within the cluster or even remotely. Such communication will most likely bypass existing security measures in the cluster. This control determines which subjects have permissions to use this command.
Summary - Passed:4   Excluded:0   Failed:0   Total:4

[control: Privileged container - https://hub.armo.cloud/docs/c-0057] passed 👍
Description: Potential attackers may gain access to privileged containers and inherit access to the host resources. Therefore, it is not recommended to deploy privileged containers unless it is absolutely necessary. This control identifies all the privileged Pods.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: Resource policies - https://hub.armo.cloud/docs/c-0009] failed 😥
Description: CPU and memory resources should have a limit set for every container or a namespace to prevent resource exhaustion. This control identifies all the Pods without resource limit definitions by checking their yaml definition file as well as their namespace LimitRange objects. It is also recommended to use ResourceQuota object to restrict overall namespace resources, but this is not verified by this control.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Deployment - proxy 
  Deployment - user-scheduler 
  Job - nfs-share-creator 
  StatefulSet - user-placeholder 
Summary - Passed:0   Excluded:0   Failed:6   Total:6
Remediation: Define LimitRange and Resource Limits in the namespace or in the deployment/POD yamls.

[control: Resources CPU limit and request - https://hub.armo.cloud/docs/c-0050] failed 😥
Description: This control identifies all Pods for which the CPU limit is not set.
Failed:
  Deployment - docs-service 
  Deployment - hub 
  Deployment - proxy 
  Deployment - user-scheduler 
  Job - nfs-share-creator 
  StatefulSet - user-placeholder 
Summary - Passed:0   Excluded:0   Failed:6   Total:6
Remediation: Set the CPU limit or use exception mechanism to avoid unnecessary notifications.

[control: Resources memory limit and request - https://hub.armo.cloud/docs/c-0004] failed 😥
Description: This control identifies all Pods for which the memory limit is not set.
Failed:
  Deployment - docs-service 
  Job - nfs-share-creator 
Summary - Passed:4   Excluded:0   Failed:2   Total:6
Remediation: Set the memory limit or use exception mechanism to avoid unnecessary notifications.

[control: SSH server running inside container - https://hub.armo.cloud/docs/c-0042] passed 👍
Description: An SSH server that is running inside a container may be used by attackers to get remote access to the container. This control checks if pods have an open SSH port (22/2222).
Summary - Passed:11   Excluded:0   Failed:0   Total:11

[control: Secret/ETCD encryption enabled - https://hub.armo.cloud/docs/c-0066] skipped 😕
Description: All Kubernetes Secrets are stored primarily in etcd therefore it is important to encrypt it.
[control: Sudo in container entrypoint - https://hub.armo.cloud/docs/c-0062] passed 👍
Description: Adding sudo to a container entry point command may escalate process privileges and allow access to forbidden resources. This control checks all the entry point commands in all containers in the POD to find those that have sudo command.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

[control: Writable hostPath mount - https://hub.armo.cloud/docs/c-0045] passed 👍
Description: Mounting host directory to the container can be used by attackers to get access to the underlying host and gain persistence.
Summary - Passed:6   Excluded:0   Failed:0   Total:6

+-----------------------------------------------------------------------+------------------+--------------------+---------------+--------------+
|                             CONTROL NAME                              | FAILED RESOURCES | EXCLUDED RESOURCES | ALL RESOURCES | % RISK-SCORE |
+-----------------------------------------------------------------------+------------------+--------------------+---------------+--------------+
| Access Kubernetes dashboard                                           |        0         |         0          |       2       |      0%      |
| Access container service account                                      |        6         |         0          |      13       |     50%      |
| Access tiller endpoint                                                |        0         |         0          |       0       |   skipped    |
| Allow privilege escalation                                            |        2         |         0          |       6       |     27%      |
| Allowed hostPath                                                      |        0         |         0          |       6       |      0%      |
| Applications credentials in configuration files                       |        1         |         0          |       9       |      9%      |
| Audit logs enabled                                                    |        0         |         0          |       0       |   skipped    |
| Automatic mapping of service account                                  |        0         |         0          |       9       |      0%      |
| CVE-2021-25741 - Using symlink for arbitrary host file system access. |        0         |         0          |       0       |   skipped    |
| CVE-2021-25742-nginx-ingress-snippet-annotation-vulnerability         |        0         |         0          |       0       |   skipped    |
| CVE-2022-0185-linux-kernel-container-escape                           |        0         |         0          |       0       |   skipped    |
| CVE-2022-0492-cgroups-container-escape                                |        2         |         0          |       6       |     27%      |
| CVE-2022-24348-argocddirtraversal                                     |        0         |         0          |       0       |   skipped    |
| Cluster internal networking                                           |        0         |         0          |       0       |   skipped    |
| Cluster-admin binding                                                 |        0         |         0          |       4       |      0%      |
| Configured liveness probe                                             |        2         |         0          |       5       |     32%      |
| Configured readiness probe                                            |        2         |         0          |       5       |     32%      |
| Container hostPort                                                    |        0         |         0          |       6       |      0%      |
| Containers mounting Docker socket                                     |        0         |         0          |       6       |      0%      |
| Control plane hardening                                               |        0         |         0          |       0       |   skipped    |
| CoreDNS poisoning                                                     |        0         |         0          |       7       |      0%      |
| Data Destruction                                                      |        5         |         0          |       5       |     100%     |
| Delete Kubernetes events                                              |        0         |         0          |       4       |      0%      |
| Disable anonymous access to Kubelet service                           |        0         |         0          |       0       |   skipped    |
| Enforce Kubelet client TLS authentication                             |        0         |         0          |       0       |   skipped    |
| Exec into container                                                   |        0         |         0          |       4       |      0%      |
| Exposed dashboard                                                     |        0         |         0          |       9       |      0%      |
| Exposed sensitive interfaces                                          |        0         |         0          |      11       |      0%      |
| Forbidden Container Registries                                        |        3         |         0          |       6       |     41%      |
| Host PID/IPC privileges                                               |        0         |         0          |       6       |      0%      |
| HostNetwork access                                                    |        0         |         0          |       6       |      0%      |
| HostPath mount                                                        |        0         |         0          |       6       |      0%      |
| Image pull policy on latest tag                                       |        0         |         0          |       6       |      0%      |
| Images from allowed registry                                          |        4         |         0          |       6       |     55%      |
| Immutable container filesystem                                        |        6         |         0          |       6       |     99%      |
| Ingress and Egress blocked                                            |        4         |         0          |       9       |     50%      |
| Insecure capabilities                                                 |        0         |         0          |       6       |      0%      |
| K8s common labels usage                                               |        6         |         0          |       6       |     100%     |
| Kubernetes CronJob                                                    |        0         |         0          |       0       |   skipped    |
| Label usage for resources                                             |        6         |         0          |       6       |     100%     |
| Linux hardening                                                       |        6         |         0          |       6       |     100%     |
| List Kubernetes secrets                                               |        3         |         0          |       5       |     50%      |
| Malicious admission controller (mutating)                             |        0         |         0          |       0       |   skipped    |
| Malicious admission controller (validating)                           |        0         |         0          |       0       |   skipped    |
| Mount service principal                                               |        0         |         0          |       6       |      0%      |
| Naked PODs                                                            |        0         |         0          |       0       |   skipped    |
| Namespace without service accounts                                    |        0         |         0          |       3       |      0%      |
| Network mapping                                                       |        0         |         0          |       0       |   skipped    |
| No impersonation                                                      |        0         |         0          |       4       |      0%      |
| Non-root containers                                                   |        2         |         0          |       6       |     27%      |
| PSP enabled                                                           |        0         |         0          |       0       |   skipped    |
| Pods in default namespace                                             |        6         |         0          |       6       |     99%      |
| Portforwarding privileges                                             |        0         |         0          |       4       |      0%      |
| Privileged container                                                  |        0         |         0          |       6       |      0%      |
| Resource policies                                                     |        6         |         0          |       6       |     100%     |
| Resources CPU limit and request                                       |        6         |         0          |       6       |     100%     |
| Resources memory limit and request                                    |        2         |         0          |       6       |     27%      |
| SSH server running inside container                                   |        0         |         0          |      11       |      0%      |
| Secret/ETCD encryption enabled                                        |        0         |         0          |       0       |   skipped    |
| Sudo in container entrypoint                                          |        0         |         0          |       6       |      0%      |
| Writable hostPath mount                                               |        0         |         0          |       6       |      0%      |
+-----------------------------------------------------------------------+------------------+--------------------+---------------+--------------+
|                           RESOURCE SUMMARY                            |        12        |         0          |      25       |    23.54%    |
+-----------------------------------------------------------------------+------------------+--------------------+---------------+--------------+
FRAMEWORKS: DevOpsBest (risk: 45.61), MITRE (risk: 13.12), ArmoBest (risk: 21.08), NSA (risk: 18.40)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Scan results have not been submitted: Kubescape will submit scan results only when scanning a cluster (not YAML files)
Sign up for free: https://portal.armo.cloud/account/login
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

choldgraf commented 2 years ago

This in general seems like a great idea to me, provided that we agree the tests they run are the "right tests" to run, and that the conclusion of those tests is reasonable. I agree with @sgibson91 that it would be compelling to be able to tell communities that we run XYZ standardize security checks on our infrastructure. We just want to make sure this doesn't commit us to super unsustainable security practices that have marginal benefit :-)

damianavila commented 2 years ago

run this function in our validation CI/CD (it's probably too verbose to also do it on deploy).

I like the idea, although maybe running it in our validation CI/CD is too much frequency? WDYT about some cron-based maybe weekly/monthly cadence, @sgibson91?

We just want to make sure this doesn't commit us to super unsustainable security practices that have marginal benefit

Very good point, @choldgraf!

sgibson91 commented 2 years ago

WDYT about some cron-based maybe weekly/monthly cadence, @sgibson91?

I think that would be fine, but I think running it in PRs that change our config (i.e., before deployment!) would be better practice. We want to catch security risks before they get into an environment where they could be harmful, right?

damianavila commented 2 years ago

We want to catch security risks before they get into an environment where they could be harmful, right?

Yep, then my next question would be: are people going to read the long output for each PR? And now another one coming to my mind, do we want to expose this information publicly without looking at it first? Are not we expanding the attack surface area if we do that?

yuvipanda commented 2 years ago

I'd assume you'd form an opinionated set of options to kubescape, fix all the errors it spits out first, and then treat any deviation as a failure.

sgibson91 commented 2 years ago

@yuvipanda yes, you can customise frameworks to achieve that opinionated set of options https://www.youtube.com/watch?v=12Sanq_rEhs

yuvipanda commented 2 years ago

https://www.paloaltonetworks.com/apps/pan/public/downloadResource?pagePath=/content/pan/en_US/resources/whitepapers/kubernetes-privilege-escalation-excessive-permissions-in-popular-platforms is another guide.

2i2c-org / infrastructure