fluxcd / flux2

Open and extensible continuous delivery solution for Kubernetes. Powered by GitOps Toolkit.
https://fluxcd.io
Apache License 2.0
6.54k stars 604 forks source link

flux v0.9.1 installation fails on openshift 4.7 #1110

Closed waseem-h closed 3 years ago

waseem-h commented 3 years ago

Describe the bug

Hi, I'm trying to install flux v0.9.1 that deploys the source-controller as well. The problem is that the fsGroup: 1337 in securityContext is hardcoded in both source-controller as well as kustomize-controller manifests. This makes it harder for us to deploy it via the flux install command directly. A workaround is to export the manifests and change the value but its hard to do that if we're installing it via a CI/CD pipeline or similar. Would it be possible to have a flag that allows to customize the initial manifests? Because it will be an egg chicken problem if I try to do it via Kustomize as both kustomie-controller and source-controller won't be running initially.

To Reproduce

Steps to reproduce the behaviour:

flux install --version v0.9.1

Expected behavior

Installation on openshift should succeed by default

Additional context

Below please provide the output of the following commands:

flux --version
root@20f9eaaf73f3:/terraform# flux --version
flux version 0.9.1

flux check
root@20f9eaaf73f3:/terraform# flux check
► checking prerequisites
✔ kubectl 1.20.1-5-g76a04fc >=1.18.0-0
✔ Kubernetes 1.20.0+5fbfd19 >=1.16.0-0
► checking controllers
✔ helm-controller: healthy
► ghcr.io/fluxcd/helm-controller:v0.8.1
✔ kustomize-controller: healthy
► ghcr.io/fluxcd/kustomize-controller:v0.9.2
✔ notification-controller: healthy
► ghcr.io/fluxcd/notification-controller:v0.9.0
✗ source-controller: unhealthy (timed out waiting for rollout)
► ghcr.io/fluxcd/source-controller:v0.9.0

kubectl -n <namespace> get all
root@20f9eaaf73f3:/terraform# kubectl -n flux-system get all
NAME                                          READY   STATUS    RESTARTS   AGE
pod/helm-controller-7fd55b8c9f-g2ltm          1/1     Running   0          7m27s
pod/kustomize-controller-84fdd79d5b-wvtht     1/1     Running   0          7m27s
pod/notification-controller-d9464dbdf-nkcvd   1/1     Running   0          7m26s

NAME                              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/notification-controller   ClusterIP   172.30.134.55   <none>        80/TCP    7m27s
service/source-controller         ClusterIP   172.30.22.141   <none>        80/TCP    7m27s
service/webhook-receiver          ClusterIP   172.30.39.33    <none>        80/TCP    7m27s

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/helm-controller           1/1     1            1           7m27s
deployment.apps/kustomize-controller      1/1     1            1           7m27s
deployment.apps/notification-controller   1/1     1            1           7m27s
deployment.apps/source-controller         0/1     0            0           7m26s

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/helm-controller-7fd55b8c9f          1         1         1       7m27s
replicaset.apps/kustomize-controller-84fdd79d5b     1         1         1       7m27s
replicaset.apps/notification-controller-d9464dbdf   1         1         1       7m26s
replicaset.apps/source-controller-798bd8fffb        1         0         0       7m26s

root@20f9eaaf73f3:/terraform# kubectl describe rs source-controller-798bd8fffb -n flux-system
Name:           source-controller-798bd8fffb
Namespace:      flux-system
Selector:       app=source-controller,pod-template-hash=798bd8fffb
Labels:         app=source-controller
                pod-template-hash=798bd8fffb
Annotations:    deployment.kubernetes.io/desired-replicas: 1
                deployment.kubernetes.io/max-replicas: 1
                deployment.kubernetes.io/revision: 1
Controlled By:  Deployment/source-controller
Replicas:       0 current / 1 desired
Pods Status:    0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:           app=source-controller
                    pod-template-hash=798bd8fffb
  Annotations:      prometheus.io/port: 8080
                    prometheus.io/scrape: true
  Service Account:  source-controller
  Containers:
   manager:
    Image:       ghcr.io/fluxcd/source-controller:v0.9.0
    Ports:       9090/TCP, 8080/TCP, 9440/TCP
    Host Ports:  0/TCP, 0/TCP, 0/TCP
    Args:
      --events-addr=http://notification-controller/
      --watch-all-namespaces=true
      --log-level=info
      --log-encoding=json
      --enable-leader-election
      --storage-path=/data
      --storage-adv-addr=source-controller.$(RUNTIME_NAMESPACE).svc.cluster.local.
    Limits:
      cpu:     1
      memory:  1Gi
    Requests:
      cpu:      50m
      memory:   64Mi
    Liveness:   http-get http://:healthz/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      RUNTIME_NAMESPACE:   (v1:metadata.namespace)
    Mounts:
      /data from data (rw)
      /tmp from tmp (rw)
  Volumes:
   data:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
   tmp:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
Conditions:
  Type             Status  Reason
  ----             ------  ------
  ReplicaFailure   True    FailedCreate
Events:
  Type     Reason        Age                    From                   Message
  ----     ------        ----                   ----                   -------
  Warning  FailedCreate  2m35s (x17 over 8m3s)  replicaset-controller  Error creating: pods "source-controller-798bd8fffb-" is forbidden: unable to validate against any security context constraint: [provider restricted: .spec.securityContext.fsGroup: Invalid value: []int64{1337}: 1337 is not an allowed group]
stefanprodan commented 3 years ago

Duplicate of: https://github.com/fluxcd/flux2/discussions/1051