Include a sample pod security policy

[pebrc]

This could be part or #2785 or a standalone sample configuration to illustrate a secure ECK setup when users have turned on the pod security policy admission controller.

[nachiket-lab]

Facing this issue, not sure what more permissions to give. I currently have a self hosted Kubernetes, with PSP enabled.

I have two PSP enabled on my cluster restricted and privileged. The restricted policy is enabled by default for all authenticated users in the cluster. I have also allowed the elastic-operator service account to use the privileged policy for a specific namespace. However, the init container still fails with the error:

 CreateContainerConfigError: container has runAsNonRoot and image will run as root 

Environment Specs

Kubernetes: v1.19.1 ECK Version: 1.2

The elasticsearch.yml used:

kubectl apply -f elasticsearch.yml -n elastic

kind: Elasticsearch
metadata:
  name: cdc
spec:
  version: 7.9.1
  nodeSets:
  - name: default
    count: 3
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: longhorn

The PSP cluster binding:

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp:privileged
rules:
- apiGroups:
  - policy
  resources:
  - podsecuritypolicies
  resourceNames:
  - z-privileged # the psp we are giving access to
  verbs:
  - use
---
# This applies psp/restricted to all authenticated users
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp:privileged
subjects:
- kind: ServiceAccount
  name: calico-kube-controllers
  namespace: kube-system
- kind: ServiceAccount
  name: calico-node
  namespace: kube-system
- kind: ServiceAccount
  name: elastic-operator
  namespace: elastic
roleRef:
  kind: ClusterRole
  name: psp:privileged
  apiGroup: rbac.authorization.k8s.io

The privileged PSP:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: z-privileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'

The init container is still failing with the error that container run as non-root is specified.

[david-kow]

Hey @NerdSec, can you check the kubernetes.io/psp annotation on the Pods? Does it contain the expected PSP?

Also, ECK operator itself is not creating the Pods, statefulset controller is. I'm not sure about Calico, but on vanilla k8s it's also needed to add the below for the right controllers to get psp use permission.

subjects:
...
- kind: Group
  name: system:serviceaccounts
  namespace: kube-system

Does calico-kube-controllers or calico-node contain statefulset controller?

Btw, we tend to use GitHub for features/bugs, so for questions like above you can use our Discuss forum.

[nachiket-lab]

Hi @david-kow ,

Sure, will shift this to Discuss.

So, calico does not use a stateful set controller to create any pods, rather, it uses two serviceaccounts:

1. calico-node creates a daemon set that runs on all the kubernetes hosts (privileged).
2. calico-kube-controller creates a scalable deployment of 1 node (privileged).

These pods in turn do not create any additional pods. So i can restrict the serviceaccounts that are allowed to use my privileged PSP in a particular namespace.

---

Also, wouldn't this piece give access to all authenticated serviceaccounts to use the privileged policy in the kube-system NS? If so, this does not seem to be necessary and too permissive.

subjects:
...
- kind: Group
  name: system:serviceaccounts
  namespace: kube-system

Discuss link: https://discuss.elastic.co/t/support-for-eck-with-a-psp/249454