search

bitnami-labs / sealed-secrets

A Kubernetes controller and tool for one-way encrypted Secrets
Apache License 2.0
7.62k stars 683 forks source link

Installing problems on openshift 4.12.5 due security context #1119

Closed aladrocMatiner closed 1 year ago

aladrocMatiner commented 1 year ago

Hej, Im running openshift 4.12.5 and Kubeseal 0.19.5 helm 3.9.0

╭─jromero at mgnt-00 in ~/infra/bin/kubeseal/0.19.5                                                                                                                                                             ╰─○ helm install jromero bitnami/sealed-secrets                                                                                                                                                                 WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jromero/infra/ose-clusters/tocp/auth/kubeconfig                                                                     NAME: jromero                                                                                                                                                                                                   LAST DEPLOYED: Thu Mar  2 17:30:20 2023                                                                                                                                                                         NAMESPACE: jromero                                                                                                                                                                                              STATUS: deployed                                                                                                                                                                                                REVISION: 1                                                                                                                                                                                                     TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **

Watch the SealedSecret controller status using the command:

    kubectl get deploy -w --namespace jromero -l app.kubernetes.io/name=sealed-secrets,app.kubernetes.io/instance=jromero

Once the controller is up and ready, you should be able to create sealed secrets.
                                                                                                                                                                                                                1. Install the client-side tool (kubeseal) as explained in the docs below:                                                                                                                                                                                                                                                                                                                                                          https://github.com/bitnami-labs/sealed-secrets#installation-from-source                                                                                                                                                                                                                                                                                                                                                     2. Create a sealed secret file running the command below:

    kubectl create secret generic secret-name --dry-run=client --from-literal=foo=bar -o yaml | \
    kubeseal \
      --controller-name=jromero-sealed-secrets \                                                                                                                                                                      --controller-namespace=jromero \                                                                                                                                                                                --format yaml > mysealedsecret.yaml

The file mysealedsecret.yaml is a commitable file.

If you would rather not need access to the cluster to generate the sealed secret you can run:

    kubeseal \
      --controller-name=jromero-sealed-secrets \
      --controller-namespace=jromero \
      --format yaml > mysealedsecret.yaml

to retrieve the public cert used for encryption and store it locally. You can then run 'kubeseal --cert mycert.pem' instead to use the local cert e.g.

    kubectl create secret generic secret-name --dry-run=client --from-literal=foo=bar -o yaml | \
    kubeseal \
      --controller-name=jromero-sealed-secrets \
      --controller-namespace=jromero \
      --format yaml --cert mycert.pem > mysealedsecret.yaml

3. Apply the sealed secret:

    kubectl create -f mysealedsecret.yaml

Running 'kubectl get secret secret-name -o yaml' will show the decrypted secret that was generated from the sealed secret.

Both the SealedSecret and generated Secret must have the same name and namespace.
╭─jromero at mgnt-00 in ~/infra/bin/kubeseal/0.19.5
╰─○ oc get all
NAME                             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/jromero-sealed-secrets   ClusterIP   172.30.152.0   <none>        8080/TCP   5m24s

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/jromero-sealed-secrets   0/1     0            0           5m24s

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/jromero-sealed-secrets-656f56648b   1         0         0       5m24s

╭─jromero at mgnt-00 in ~/infra/bin/kubeseal/0.19.5
╰─○ oc get events
LAST SEEN   TYPE      REASON              OBJECT                                            MESSAGE
9m5s        Warning   FailedCreate        replicaset/jromero-sealed-secrets-656f56648b      Error creating: pods "jromero-sealed-secrets-656f56648b-" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{1001}: 1001 is not an allowed group, spec.containers[0].securityContext.runAsUser: Invalid value: 1001: must be in the ranges: [1000680000, 1000689999], provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]
12s         Warning   FailedCreate        replicaset/jromero-sealed-secrets-656f56648b      Error creating: pods "jromero-sealed-secrets-656f56648b-" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{1001}: 1001 is not an allowed group, spec.containers[0].securityContext.runAsUser: Invalid value: 1001: must be in the ranges: [1000680000, 1000689999], provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]
11m         Normal    ScalingReplicaSet   deployment/jromero-sealed-secrets                 Scaled up replica set jromero-sealed-secrets-656f56648b to 1
5m40s       Normal    ScalingReplicaSet   deployment/jromero-sealed-secrets                 Scaled up replica set jromero-sealed-secrets-656f56648b to 1
19m         Warning   FailedCreate        replicaset/lalala-sealed-secrets-7c947b78b9       Error creating: pods "lalala-sealed-secrets-7c947b78b9-" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{1001}: 1001 is not an allowed group, spec.containers[0].securityContext.runAsUser: Invalid value: 1001: must be in the ranges: [1000680000, 1000689999], provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]
43m         Normal    ScalingReplicaSet   deployment/lalala-sealed-secrets                  Scaled up replica set lalala-sealed-secrets-7c947b78b9 to 1
107m        Warning   FailedCreate        replicaset/my-release-sealed-secrets-7bcd7778f6   Error creating: pods "my-release-sealed-secrets-7bcd7778f6-" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{1001}: 1001 is not an allowed group, spec.containers[0].securityContext.runAsUser: Invalid value: 1001: must be in the ranges: [1000680000, 1000689999], provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]
49m         Warning   FailedCreate        replicaset/my-release-sealed-secrets-7bcd7778f6   Error creating: pods "my-release-sealed-secrets-7bcd7778f6-" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations[seccomp.security.alpha.kubernetes.io/pod]: Forbidden: seccomp may not be set, pod.metadata.annotations[container.seccomp.security.alpha.kubernetes.io/sealed-secrets]: Forbidden: seccomp may not be set, provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{1001}: 1001 is not an allowed group, spec.containers[0].securityContext.runAsUser: Invalid value: 1001: must be in the ranges: [1000680000, 1000689999], provider "restricted": Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": Forbidden: not usable by user or serviceaccount, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]
123m        Normal    ScalingReplicaSet   deployment/my-release-sealed-secrets              Scaled up replica set my-release-sealed-secrets-7bcd7778f6 to 1

There is a way to deal with scc and anyuid on the install?

agarcia-oss commented 1 year ago

Hi @aladrocMatiner We test the controller installation in Openshift using the following values:

containerSecurityContext:
  enabled: true
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: null
podSecurityContext:
  enabled: false

Can you try them and let us know if this works for you? Regards

github-actions[bot] commented 1 year ago

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

github-actions[bot] commented 1 year ago

Due to the lack of activity in the last 7 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.