ArgoCD fails to manage ClusterRole and ClusterRoleBindings

[eshtranvasser]

ArgoCD version: v2.10.2+fcf5d8c Kubernetes version: 1.27.7

OpenTelemetry operator allows us to create Collector with OpenTelemetryCollector CRD. However, it does not allow to define ClusterRole and ClusterRole binding. Our motivation was to add those resources using Argo, as one of the manifests.

Trying to define ClusterRole using source git:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: opentelemetry-coralogix-collector
rules:
  - apiGroups: [""]
    resources: ["pods", "namespaces", "nodes"]
    verbs: ["get", "watch", "list"]
  - apiGroups: ["apps"]
    resources: ["replicasets"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["extensions"]
    resources: ["replicasets"]
    verbs: ["get", "list", "watch"]

Here is the Application used:

apiVersion: argoproj.io/v1alpha1
kind: Application
  name: my_application
  namespace: argo-system
spec:
  destination:
    namespace: '*'
    server: https://kubernetes.default.svc
  project: my_project
  source:
    path: my_path
    repoURL: ***.git
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Here is the project used:

apiVersion: argoproj.io/v1alpha1
kind: AppProject
  name: my_project
  namespace: argo-system
spec:
  clusterResourceWhitelist:
  - group: '*'
    kind: '*'
  description: 'Stack'
  destinations:
  - namespace: '*'
    server: https://kubernetes.default.svc
  orphanedResources:
    warn: false
  sourceRepos:
  - '*'

ClusterRole fails to sync with exception: error running rbacReconcile: error running kubectl auth reconcile: namespaces \"*\" not found

Application controller log: time="2024-05-20T09:13:59Z" level=info msg="Applying resource ClusterRole/opentelemetry-coralogix-collector in cluster: https://10.0.0.1:443, namespace: *" dry-run=none manager=argocd-controller serverSideApply=true serverSideDiff=false time="2024-05-20T09:13:59Z" level=info msg="{\"apiVersion\":\"rbac.authorization.k8s.io/v1\",\"kind\":\"ClusterRole\",\"metadata\":{\"annotations\":{\"argocd.argoproj.io/sync-options\":\"ServerSideApply=true\"},\"labels\":{\"argocd.argoproj.io/instance\":\"***\"},\"name\":\"opentelemetry-coralogix-collector\",\"namespace\":\"*\"},\"rules\":[{\"apiGroups\":[\"\"],\"resources\":[\"pods\",\"namespaces\",\"nodes\"],\"verbs\":[\"get\",\"watch\",\"list\"]},{\"apiGroups\":[\"apps\"],\"resources\":[\"replicasets\"],\"verbs\":[\"get\",\"list\",\"watch\"]},{\"apiGroups\":[\"extensions\"],\"resources\":[\"replicasets\"],\"verbs\":[\"get\",\"list\",\"watch\"]}]}" time="2024-05-20T09:13:59Z" level=info msg="Apply failed" application=argo-system/*** dryRun=false message="error running rbacReconcile: error running kubectl auth reconcile: namespaces \"*\" not found" syncId=00110-RYDTD task="Sync/0 resource rbac.authorization.k8s.io/ClusterRole:*/opentelemetry-coralogix-collector nil->obj (,,)" time="2024-05-20T09:13:59Z" level=info msg="Adding resource result, status: 'SyncFailed', phase: 'Failed', message: 'error running rbacReconcile: error running kubectl auth reconcile: namespaces \"*\" not found'" application=argo-system/*** kind=ClusterRole name=opentelemetry-coralogix-collector namespace="*" phase=Sync syncId=00110-RYDTD

As we see from the logs, during the reconciliation namespace: "*" is injected into the ClusterRole manifest. After that, controller runs kubectl auth reconcile command on this manifest and fails.

Running kubectl auth reconcile works if running locally. This is also works if ClusterRole is part of Helm Chart.

However, managing RBAC resources seems to be tricky with ArgoCD. I tried different annotations and ArgoCD configurations, but without success. This behaviour seems like bug, because managing ClusterRole and ClusterRoleBinding seems to be very basic feature.

Is there any option to disable kubectl auth reconcile during Sync on specific resource? Is there any option to disable injecting namespace: "*" to the cluster wide manifests.

[eshtranvasser]

Any update on this one? Why ArgoCD cannot be used for creating ClusterRole and ClusterRoleBinding? What am I missing? Thanks in advance.

[raynay-r]

@eshtranvasser I am having the same issue. Did you find a solution?

[raynay-r]

For me it was just removing namespace: * from the application manifest.

So instead of:

destination:
  server: 'https://kubernetes.default.svc'
  namespace: '*'
  name: in-cluster

just use the following:

destination:
  server: 'https://kubernetes.default.svc'
  name: in-cluster