search

Shopify / kubeaudit

kubeaudit helps you audit your Kubernetes clusters against common security controls
MIT License
1.89k stars 183 forks source link

Kubeaudit not recognizing pods running as root #476

Closed Tim-Schwalbe closed 2 years ago

Tim-Schwalbe commented 2 years ago
ISSUE TYPE

Bug Report

BUG REPORT

SUMMARY

Just see at the end my script running in gitlab-runner inside the cluster

ENVIRONMENT

https://github.com/Shopify/kubeaudit/releases/download/v0.19.0/kubeaudit_0.19.0_linux_amd64.tar.gz K8S: 1.24

STEPS TO REPRODUCE

I have for sure an application pod running as root, privileged and non read-only filesystem.

When I run kubeaudit inside a gitlab-runner the cluster recognizes only the runner pod itself, but not the app pod.

inside app container:

image

EXPECTED RESULTS

I thought it would scan all running pods for vulnerabilities

ACTUAL RESULTS

The output only shows missing namespace NetworkPolicies and the runner pod itself as failed.

---------------- Results for --------------- apiVersion: v1 kind: Namespace metadata: name: default

-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy Message: Namespace is missing a default deny ingress and egress NetworkPolicy. Metadata: Namespace: default

ADDITIONAL INFORMATION
$ wget ${KUBE_AUDIT_DOWNLOAD_URL} && tar -zxf kubeaudit_*.tar.gz kubeaudit && chown root:root kubeaudit
Connecting to github.com (140.82.121.3:443)
Connecting to objects.githubusercontent.com (185.[19](https://gitlab.com/9#L19)9.108.133:443)
saving to 'kubeaudit_0.19.0_linux_amd64.tar.gz'
kubeaudit_0.19.0_lin 100% |********************************| 9526k  0:00:00 ETA
'kubeaudit_0.19.0_linux_amd64.tar.gz' saved
$ tar_name=$(ls kubeaudit_*.tar.gz) && echo "${CHECKSUM}  $tar_name" | sha256sum -c
kubeaudit_0.19.0_linux_amd64.tar.gz: OK
$ ./kubeaudit all
" level=info msg="Running inside cluster, using the cluster config"
---------------- Results for ---------------
  apiVersion: v1
  kind: Pod
  metadata:
    name: runner-uxmmvxa-project-concurrent-0hpjxm
    namespace: tools
--------------------------------------------
-- [error] AppArmorAnnotationMissing
   Message: AppArmor annotation missing. The annotation 'container.apparmor.security.beta.kubernetes.io/build' should be added.
   Metadata:
      Container: build
      MissingAnnotation: container.apparmor.security.beta.kubernetes.io/build
-- [error] AppArmorAnnotationMissing
   Message: AppArmor annotation missing. The annotation 'container.apparmor.security.beta.kubernetes.io/helper' should be added.
   Metadata:
      Container: helper
      MissingAnnotation: container.apparmor.security.beta.kubernetes.io/helper
-- [error] AppArmorAnnotationMissing
   Message: AppArmor annotation missing. The annotation 'container.apparmor.security.beta.kubernetes.io/init-permissions' should be added.
   Metadata:
      Container: init-permissions
      MissingAnnotation: container.apparmor.security.beta.kubernetes.io/init-permissions
-- [error] CapabilityShouldDropAll
   Message: Capability Drop list should be set to ALL. Add the specific ones you need to the Add list and set an override label.
   Metadata:
      Container: build
-- [error] CapabilityShouldDropAll
   Message: Capability Drop list should be set to ALL. Add the specific ones you need to the Add list and set an override label.
   Metadata:
      Container: helper
-- [error] CapabilityOrSecurityContextMissing
   Message: Security Context not set. The Security Context should be specified and all Capabilities should be dropped by setting the Drop list to ALL.
   Metadata:
      Container: init-permissions
-- [warning] LimitsNotSet
   Message: Resource limits not set.
   Metadata:
      Container: build
-- [warning] LimitsNotSet
   Message: Resource limits not set.
   Metadata:
      Container: helper
-- [warning] LimitsNotSet
   Message: Resource limits not set.
   Metadata:
      Container: init-permissions
-- [error] RunAsNonRootPSCNilCSCNil
   Message: runAsNonRoot should be set to true or runAsUser should be set to a value > 0 either in the container SecurityContext or PodSecurityContext.
   Metadata:
      Container: build
-- [error] RunAsNonRootPSCNilCSCNil
   Message: runAsNonRoot should be set to true or runAsUser should be set to a value > 0 either in the container SecurityContext or PodSecurityContext.
   Metadata:
      Container: helper
-- [error] RunAsNonRootPSCNilCSCNil
   Message: runAsNonRoot should be set to true or runAsUser should be set to a value > 0 either in the container SecurityContext or PodSecurityContext.
   Metadata:
      Container: init-permissions
-- [error] AllowPrivilegeEscalationNil
   Message: allowPrivilegeEscalation not set which allows privilege escalation. It should be set to 'false'.
   Metadata:
      Container: build
-- [error] AllowPrivilegeEscalationNil
   Message: allowPrivilegeEscalation not set which allows privilege escalation. It should be set to 'false'.
   Metadata:
      Container: helper
-- [error] AllowPrivilegeEscalationNil
   Message: allowPrivilegeEscalation not set which allows privilege escalation. It should be set to 'false'.
   Metadata:
      Container: init-permissions
-- [error] PrivilegedTrue
   Message: privileged is set to 'true' in container SecurityContext. It should be set to 'false'.
   Metadata:
      Container: build
-- [error] PrivilegedTrue
   Message: privileged is set to 'true' in container SecurityContext. It should be set to 'false'.
   Metadata:
      Container: helper
-- [warning] PrivilegedNil
   Message: privileged is not set in container SecurityContext. Privileged defaults to 'false' but it should be explicitly set to 'false'.
   Metadata:
      Container: init-permissions
-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: build
-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: helper
-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: init-permissions
-- [error] SeccompAnnotationMissing
   Message: Seccomp annotation is missing. The annotation seccomp.security.alpha.kubernetes.io/pod: runtime/default should be added.
   Metadata:
      MissingAnnotation: seccomp.security.alpha.kubernetes.io/pod
---------------- Results for ---------------
  apiVersion: v1
  kind: Pod
  metadata:
    name: runner-uxmmvxa-project
    namespace: tools
--------------------------------------------
-- [error] AppArmorAnnotationMissing
   Message: AppArmor annotation missing. The annotation 'container.apparmor.security.beta.kubernetes.io/build' should be added.
   Metadata:
      Container: build
      MissingAnnotation: container.apparmor.security.beta.kubernetes.io/build
-- [error] AppArmorAnnotationMissing
   Message: AppArmor annotation missing. The annotation 'container.apparmor.security.beta.kubernetes.io/helper' should be added.
   Metadata:
      Container: helper
      MissingAnnotation: container.apparmor.security.beta.kubernetes.io/helper
-- [error] AppArmorAnnotationMissing
   Message: AppArmor annotation missing. The annotation 'container.apparmor.security.beta.kubernetes.io/init-permissions' should be added.
   Metadata:
      Container: init-permissions
      MissingAnnotation: container.apparmor.security.beta.kubernetes.io/init-permissions
-- [error] CapabilityShouldDropAll
   Message: Capability Drop list should be set to ALL. Add the specific ones you need to the Add list and set an override label.
   Metadata:
      Container: build
-- [error] CapabilityShouldDropAll
   Message: Capability Drop list should be set to ALL. Add the specific ones you need to the Add list and set an override label.
   Metadata:
      Container: helper
-- [error] CapabilityOrSecurityContextMissing
   Message: Security Context not set. The Security Context should be specified and all Capabilities should be dropped by setting the Drop list to ALL.
   Metadata:
      Container: init-permissions
-- [warning] LimitsNotSet
   Message: Resource limits not set.
   Metadata:
      Container: build
-- [warning] LimitsNotSet
   Message: Resource limits not set.
   Metadata:
      Container: helper
-- [warning] LimitsNotSet
   Message: Resource limits not set.
   Metadata:
      Container: init-permissions
-- [error] RunAsNonRootPSCNilCSCNil
   Message: runAsNonRoot should be set to true or runAsUser should be set to a value > 0 either in the container SecurityContext or PodSecurityContext.
   Metadata:
      Container: build
-- [error] RunAsNonRootPSCNilCSCNil
   Message: runAsNonRoot should be set to true or runAsUser should be set to a value > 0 either in the container SecurityContext or PodSecurityContext.
   Metadata:
      Container: helper
-- [error] RunAsNonRootPSCNilCSCNil
   Message: runAsNonRoot should be set to true or runAsUser should be set to a value > 0 either in the container SecurityContext or PodSecurityContext.
   Metadata:
      Container: init-permissions
-- [error] AllowPrivilegeEscalationNil
   Message: allowPrivilegeEscalation not set which allows privilege escalation. It should be set to 'false'.
   Metadata:
      Container: build
-- [error] AllowPrivilegeEscalationNil
   Message: allowPrivilegeEscalation not set which allows privilege escalation. It should be set to 'false'.
   Metadata:
      Container: helper
-- [error] AllowPrivilegeEscalationNil
   Message: allowPrivilegeEscalation not set which allows privilege escalation. It should be set to 'false'.
   Metadata:
      Container: init-permissions
-- [error] PrivilegedTrue
   Message: privileged is set to 'true' in container SecurityContext. It should be set to 'false'.
   Metadata:
      Container: build
-- [error] PrivilegedTrue
   Message: privileged is set to 'true' in container SecurityContext. It should be set to 'false'.
   Metadata:
      Container: helper
-- [warning] PrivilegedNil
   Message: privileged is not set in container SecurityContext. Privileged defaults to 'false' but it should be explicitly set to 'false'.
   Metadata:
      Container: init-permissions
-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: build
-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: helper
-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: init-permissions
-- [error] SeccompAnnotationMissing
   Message: Seccomp annotation is missing. The annotation seccomp.security.alpha.kubernetes.io/pod: runtime/default should be added.
   Metadata:
      MissingAnnotation: seccomp.security.alpha.kubernetes.io/pod
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: cert-manager
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: cert-manager
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: default
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: default
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: develop
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: develop
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: keycloak
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: keycloak
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: kube-node-lease
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: kube-node-lease
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: kube-public
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: kube-public
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: kube-system
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: kube-system
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: monitoring
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: monitoring
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: prod
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: prod
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: staging
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: staging
---------------- Results for ---------------
  apiVersion: v1
  kind: Namespace
  metadata:
    name: tools
--------------------------------------------
-- [error] MissingDefaultDenyIngressAndEgressNetworkPolicy
   Message: Namespace is missing a default deny ingress and egress NetworkPolicy.
   Metadata:
      Namespace: tools
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1
ghost commented 2 years ago

Thanks for opening your first issue here! Be sure to follow the issue template!

Tim-Schwalbe commented 2 years ago

This did the trick for me: 

- cat ~/.kube/config > test
- ./kubeaudit all --kubeconfig "test"