hashicorp / consul-k8s

First-class support for Consul Service Mesh on Kubernetes
https://www.consul.io/docs/k8s
Mozilla Public License 2.0
670 stars 324 forks source link

Requesting support for 'restricted' PodSecurityAdmission namespace label in the Consul connect injector sidecar containers #4441

Open tspearconquest opened 2 hours ago

tspearconquest commented 2 hours ago

Community Note


Is your feature request related to a problem? Please describe.

When installing Consul 1.3.1 into a cluster running Kubernetes 1.25 or later with CNI and ConnectInject enabled, and with the application services we run making use of the Pod Security Admission feature via the namespace label pod-security.kubernetes.io/enforce: "restricted" we are unable to deploy our application services with envoy proxy/connect injector sidecars due to missing seccompProfile field under pod spec containers.securityContext field. This is visible in the pod YAML:

    securityContext:
      appArmorProfile:
        type: RuntimeDefault
      readOnlyRootFilesystem: true
      runAsGroup: 5995
      runAsNonRoot: true
      runAsUser: 5995

To be clear, this is NOT consul service pods, this is our application pods which are being injected with the Consul envoy proxy sidecar. Our pod manifests have the workload containers defined with seccompProfile field but the envoy proxy sidecar container is missing it, as this seems not to be in the code for the injected container.

This forces us to downgrade the namespace label from "restricted" value to "baseline" value until Consul supports any/all of the missing fields.

Feature Description

As I can see that the sidecar proxy containers already have appArmorProfile field set to type: RuntimeDefault I will request for the seccompProfile field to also be set to the same type: RuntimeDefault key and value pair.

Use Case(s)

Allows for Consul to run (with CNI enabled) in clusters where namespaces are configured for the restricted policy.

Contributions

Maybe but my Go is not very good so it would take a long time and likely be very poor quality code. I would recommend for someone else to tackle this.

tspearconquest commented 2 hours ago

The error we get actually covers several issues; some which will be easier to solve (such as the seccompProfile field) and some which won't (privileges/capabilities, given that the CNI is enabled and these should already have lower privileges/caps):

Warning  FailedCreate  9s (x4 over 27s)  replicaset-controller  (combined from similar events): Error creating: pods my-pod-5579699465-bf4tm" is forbidden: violates PodSecurity "restricted:latest":
  allowPrivilegeEscalation != false (containers "consul-connect-inject-init",
  "consul-dataplane" must set securityContext.allowPrivilegeEscalation=false),
  unrestricted capabilities (container "consul-dataplane" must set securityContext.capabilities.drop=["ALL"]),
  seccompProfile (pod or containers "consul-connect-inject-init",
  "consul-dataplane" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")