[RFE] Add support for PodSecurityAdmissionConfigurationTemplate on K8s 1.26 clusters

[git-ival]

Currently the TFP RKE does not support setting PSA configuration on the creation of K8s 1.26 clusters.

In order to accomplish this it seems that a few things are needed:

1. Update RKE to 1.4.4+ 1.4.8
   • There is an open PR for this: https://github.com/rancher/terraform-provider-rke/pull/397
2. Add the PSA-related options to the kube_api service block defined in the RKE documentation
   • https://rke.docs.rancher.com/config-options/services/pod-security-admission

[slickwarren]

TODO for QA: create TFP automation ticket for PSACT

[a-blender]

Due to the reasons outlined here, we will only be adding PSACT support for Method 1 (set pod_security_configuration) and Method 3 (use extra_args to pass a custom Admission Configuration file) for the provider. No customers are urgently asking for this and the latest version will be released OOB.

[a-blender]

QA Test Template

Problem

Currently the TFP RKE does not support setting PSA configuration on creation of K8s 1.25+ clusters.

Solution

• [x] 1. Update RKE to 1.4.10
• [x] 2. Add the PSA-related options to the kube_api service block defined in the RKE docs method 1 and 3.
• [x] Update PSACT docs and add/update config examples

Testing

Engineering Testing

Manual Testing

Provision an RKE cluster with TF RKE provider v.1.4.3-rc1

• [ ] Method 1 - Pass restricted policy via the kube-api.pod_security_configuration field

services {
    kube_api {
      pod_security_configuration = "<value>" # privileged or restricted
    }
  }

Deploy this yaml and verify the pod is rejected.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
    - image: nginx
      name: nginx
      ports:
        - containerPort: 80

Pass invalid value pod_security_configuration: INVALID and confirm provider throws an error.

• [ ] Method 3: Pass the path to an admission configuration file via kube-api.extra_args.admission-control-config-file and set extra_mounts to mount that file into the kube API server.

kube_api {
    extra_args = {
      admission-control-config-file: "<container-path>/admission.yaml" // path in kube API server container
    }
    extra_binds = ["<node-path>:<container-path>"]
  }

Per method 3, per the docs if the bind mount file does not exist on the node, Docker creates it as a directory. In order to use kube API extra_args to mount a custom admission configuration file to the API server you must update the Dockerfile for the API server container to touch an empty file. Otherwise, verifying the provider passes the admission-control-config-file to the node correctly is acceptable. If deployed, deploy the yaml above and verify the pod is rejected.

Automated Testing

QA Testing Considerations

Regressions Considerations

TF RKE provider node provisioning/hardened clusters.

[a-blender]

We have moved our TFP RKE test process to use RCs like Rancher and the TF rancher2 provider. Please test this issue on RC v1.4.3-rc1. I will create additional RCs for all new code in scope.

[zube[bot]]

thaneunsoo said: Tested this issue on v1.4.3-rc1 with the following scenarios

1. Pass restricted policy via the kube-api.pod_security_configuration field Deploy this yaml and verify the pod is rejected. Pass

   apiVersion: v1
   kind: Pod
   metadata:
   name: nginx
   spec:
   containers:
   - image: nginx
     name: nginx
     ports:
       - containerPort: 80

2. Pass invalid value pod_security_configuration: INVALID and confirm provider throws an error. Pass

3. Pass the path to an admission configuration file via kube-api.extra_args.admission-control-config-file and set extra_mounts to mount that file into the kube API server. Pass