Flux installation with Openshift and OLM into a dedicated namespace

blezoray commented 5 months ago

Hello, I created the following issue to flux repo but it seems for Openshift deployment, it should be there.

Issue: https://github.com/fluxcd/flux2/issues/4816

Describe the bug

Hello,

I installed Flux v2.3.0 on my lab Openshift 4.14 platform into a dedicated namespace, mpms-fluxcd-operator, using Operator Livecycle Manager (OLM).

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: fluxcd-operator
  namespace: mpms-fluxcd-operator
spec:
  channel: stable
  installPlanApproval: Manual
  name: flux
  source: community-operators
  sourceNamespace: openshift-marketplace
  startingCSV: flux.v2.3.0

But all the controllers have the default namespace flux-system hard-coded in their deployment args --events-addr=http://notification-controller.flux-system.svc.cluster.local./

      labels:
        app: kustomize-controller
    spec:
      containers:
      - args:
        - --events-addr=http://notification-controller.flux-system.svc.cluster.local./
        - --watch-all-namespaces=true
        - --log-level=info
        - --log-encoding=json
        - --enable-leader-election

You should use the variable RUNTIME_NAMESPACE or replace the url by --events-addr=http://notification-controller/

Steps to reproduce

Create a dedicated namespace, different to flux-system

Create an OperatorGroup to allow user to deploy operator in the namespace with OLM

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: fluxcd-operator
spec:
targetNamespaces: []

Create a Subscription

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: fluxcd-operator
spec:
channel: stable
installPlanApproval: Manual
name: flux
source: community-operators
sourceNamespace: openshift-marketplace
startingCSV: flux.v2.3.0

Check the deployments

Expected behavior

Args should be : - --events-addr=http://notification-controller.mpms-fluxcd-operator.svc.cluster.local./

Screenshots and recordings

No response

OS / Distro

OpenShift 4.14

Flux version

2.3.0

Flux check

no more details

Git provider

No response

Container Registry provider

No response

Additional context

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct

stefanprodan commented 5 months ago

The new Flux Operator allows setting a different namespace than flux-system, just make sure to create the FluxInstance custom resource in the same namespace where the operator has been installed. https://operatorhub.io/operator/flux-operator

blezoray commented 5 months ago

Good. Does it support multi-tenancy ?

stefanprodan commented 5 months ago

Does it support multi-tenancy ?

Yes, see the docs here: https://fluxcd.control-plane.io/operator/fluxinstance/#cluster-configuration

blezoray commented 4 months ago

Is there a way to specify the name of the ServiceAccount because the default SA has no permissions and I not sure it is recommended to change its permissions. So, I prefer to create a dedicated SA in each namespace with the associated role/rolebinding.

stefanprodan commented 4 months ago

The dedicated SA should be specified on the Flux Kustomization/HelmRelease objects under .spec.serviceAccountName, the default SA works the same as for any Kubernetes workloads, it grants no perms and it should be left like this.

Docs:

You can also change the default with a patch like so:

apiVersion: fluxcd.controlplane.io/v1
kind: FluxInstance
spec:
  kustomize:
    patches:
      - patch: |
          - op: add
            path: /spec/template/spec/containers/0/args/-
            value: --default-service-account=flux
        target:
          kind: Deployment
          name: "(kustomize-controller|helm-controller)"

blezoray commented 4 months ago

It tested the patch and it didn't replace the arg, it adds:

  spec:
    containers:
      - args:
          - --events-addr=http://notification-controller.mpms-flux-operator.svc.cluster.local./
          - --watch-all-namespaces=true
          - --log-level=info
          - --log-encoding=json
          - --enable-leader-election
          - --no-cross-namespace-refs=true
          - --no-remote-bases=true
          - --default-service-account=default
          - --concurrent=10
          - --requeue-dependency=5s
          - --default-service-account=flux

In the logs, I can see it uses the last args system:serviceaccount:mpms-ubly7356:flux Maybe, it should interesting to add a new parameter multitenantServiceAccountName` to avoid any confusion.

stefanprodan commented 4 months ago

This how it suppose to work, Flux flags support overriding, last one listed wins.

stefanprodan commented 4 months ago

Anyway, let's move this conversation to the flux-operator repo.

blezoray commented 4 months ago

I created this issue https://github.com/controlplaneio-fluxcd/flux-operator/issues/44

weaveworks / flux2-openshift