Enforce Pod Security Standard restricted

[collivier]

Description

https://kubernetes.io/docs/tutorials/security/cluster-level-pss/

Issues:

close: #1887

How has this been tested:

• [X] Covered by existing integration testing
• [ ] Added integration testing to cover
• [ ] Verified all A/C passes
  • [ ] develop
  • [ ] master
  • [ ] tag/other branch
• [ ] Test environment
  • [ ] Shared Packet K8s cluster
  • [ ] New Packet K8s cluster
  • [ ] Kind cluster
• [ ] Have not tested

[kosstennbl]

For ease of review:

TLDR: add PodSecurity: restricted as ClusterConfiguration to cluster.yml in github actions.

This PR creates file /tmp/pss/cluster-level-pss.yaml with contents:

apiVersion: apiserver.config.k8s.io/v1
        kind: AdmissionConfiguration
        plugins:
        - name: PodSecurity
          configuration:
            apiVersion: pod-security.admission.config.k8s.io/v1
            kind: PodSecurityConfiguration
            defaults:
              enforce: "restricted"
              enforce-version: "latest"
              audit: "restricted"
              audit-version: "latest"
              warn: "restricted"
              warn-version: "latest"
            exemptions:
              usernames: []
              runtimeClasses: []
              namespaces:
                - kube-system
                - local-path-storage

Then, during "Mirror setup", "sysctls specs kind config override" and "Mirror override" steps - it modifies the creation of cluster.yml: file cluster-level-pss.yaml is mounted as extra mount and used as ClusterConfiguration. Also, image: kindest/node:v1.29.2@sha256:51a1434a5397193442f0be2a297b488b6c919ce8a3931be0ce822606ea5ca245 lines are removed from cluster.yml, intentions for which are not very clear.

[collivier]

Also, image: kindest/node:v1.29.2@sha256:51a1434a5397193442f0be2a297b488b6c919ce8a3931be0ce822606ea5ca245 lines are removed from cluster.yml, intentions for which are not very clear.

It was discussed in the previous PR that double (image) pinning was useless as kind is already pinned. this will be precised in a second pending commit

[kosstennbl]

I don't have too much experience with Kubernetes and Helm and trying to review changes like this is new for me. Sorry if some of the questions are obvious or aren't making much sense.

1. It seems that chaos and some other tests aren't covered by this change as they have their own creation of cluster.yml. Is it intentional?
2. It's not quite clear to me how does this change help with functest issue, it seems that functest patches to the "baseline" security level, and it patches completely different namespaces. Litmus and cnf-testsuite namespaces are created during execution of cnf-testsuite tasks, and, to me - it seems, that they won't have any PodSecurity admission controls when tested with this change.
3. I'm not sure, but maybe we could test functionality of testsuite with restricted PodSecurity as a separate job in a pipeline, not in "spec" job? It seems that controls, that are being done with this restricted policy - are colliding with some of the testsuite tests, and even through namespaces afflicted shouldn't have recources of installed cnf, I'm still worried that some of the spec test results could be unclear or incorrect.

It was discussed in the previous PR that double (image) pinning was useless as kind is already pinned. this will be precised in a second pending commit

Couldn't find the discussion, can you link it please?