search

IBM / cloud-pak-deployer

Configuration-based installation of OpenShift and Cloud Pak for Data/Integration/Watson AIOps on various private and public cloud infrastructure providers. Deployment attempts to achieve the end-state defined in the configuration. If something fails along the way, you only need to restart the process to continue the deployment.
https://ibm.github.io/cloud-pak-deployer/
Apache License 2.0
137 stars 67 forks source link

CP4D installation on ROKS 4.14 fails #617

Closed fketelaars closed 8 months ago

fketelaars commented 8 months ago

Describe the bug When installing CP4D on ROKS 4.14, no pods appear in ibm-cert-manager. This seems to be related to a pod security settings that is different on ROKS than on other infrastructure.

On AWS

oc describe ns cs-control
Name:         cs-control
Labels:       kubernetes.io/metadata.name=cs-control
              pod-security.kubernetes.io/audit=restricted
              pod-security.kubernetes.io/audit-version=v1.24
              pod-security.kubernetes.io/warn=restricted
              pod-security.kubernetes.io/warn-version=v1.24
Annotations:  openshift.io/sa.scc.mcs: s0:c26,c25
              openshift.io/sa.scc.supplemental-groups: 1000700000/10000
              openshift.io/sa.scc.uid-range: 1000700000/10000
Status:       Active

No resource quota.

No LimitRange resource.

On ROKS:

Name:         ibm-cert-manager
Labels:       kubernetes.io/metadata.name=ibm-cert-manager
              pod-security.kubernetes.io/audit=restricted
              pod-security.kubernetes.io/audit-version=v1.24
              pod-security.kubernetes.io/enforce=restricted
              pod-security.kubernetes.io/enforce-version=v1.24
              pod-security.kubernetes.io/warn=restricted
              pod-security.kubernetes.io/warn-version=v1.24
Annotations:  openshift.io/sa.scc.mcs: s0:c26,c10
              openshift.io/sa.scc.supplemental-groups: 1000670000/10000
              openshift.io/sa.scc.uid-range: 1000670000/10000
Status:       Active

No resource quota.

No LimitRange resource.

Workaround Remove the enforced pod security from the namespaces created by CP4D:

There are two steps, documented here: https://docs.openshift.com/container-platform/4.11/authentication/understanding-and-managing-pod-security-admission.html#security-context-constraints-psa-opting_understanding-and-managing-pod-security-admission

  1. Disable synchronization:

    export PSP_NS=ibm-cert-manager
oc label ns $PSP_NS security.openshift.io/scc.podSecurityLabelSync=false

  2. Remove enforce labels:

    oc label ns $PSP_NS pod-security.kubernetes.io/enforce-version- pod-security.kubernetes.io/enforce-

Solution

fketelaars commented 8 months ago

With the latest ROKS deployment the labels no longer cause CP4D installs to fail:

oc describe ns ibm-cert-manager
Name:         ibm-cert-manager
Labels:       kubernetes.io/metadata.name=ibm-cert-manager
              pod-security.kubernetes.io/audit=restricted
              pod-security.kubernetes.io/audit-version=v1.24
              pod-security.kubernetes.io/enforce=privileged
              pod-security.kubernetes.io/warn=restricted
              pod-security.kubernetes.io/warn-version=v1.24
Annotations:  openshift.io/sa.scc.mcs: s0:c26,c5
              openshift.io/sa.scc.supplemental-groups: 1000660000/10000
              openshift.io/sa.scc.uid-range: 1000660000/10000
Status:       Active

No resource quota.

No LimitRange resource.