Okta group impersonate limitation

knopka commented 2 years ago

Describe the bug

Looks like this configuration https://github.com/weaveworks/weave-gitops/tree/main/charts/gitops-server#impersonate doesn't work

rbac:
  create: true
  impersonationResources: ["groups"]
  impersonationResourceNames: ["gitops-reader"]

I have configured login via an OIDC(Okta) provider. In Okta, group created and user added to this group. Okta Application and Authorization Server configured as well. Then I created ClusterRole and ClusterRoleBinding with subject as group name I created in Okta. Without rbac configuration block I mentioned above all works and I can see cluster data in UI, but after adding rbac configuration block UI is empty.

From Logs

2022-09-04T06:44:42.172Z    DEBUG   gitops.auth-server  auth/jwt.go:155 Found principal {"user": "raman", "groups": ["wego-admin-rk"], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2022-09-04T06:44:42.172Z    DEBUG   gitops.auth-server  auth/jwt.go:155 Found principal {"user": "raman", "groups": ["wego-admin-rk"], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2022-09-04T06:44:42.194Z    ERROR   gitops  clustersmngr/factory.go:367 failed filtering namespaces {"cluster": "Default", "user": "raman", "error": "user namespace access: users \"raman\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops-weave-gitops\" cannot impersonate resource \"users\" in API group \"\" at the cluster scope"}

Environment

Weave-Gitops Version - helm chart version - 3.0.3
Flux Version - v0.31.2
Kubernetes version - v1.21.14-eks-18ef993

To Reproduce Steps to reproduce the behavior:

Configure Okta (User, Group, Application and Authorization Server)
Create ClusterRole and ClusterRoleBinding with subject as group name created in Okta.

Add configuration below:

rbac:
create: true
impersonationResources: ["groups"]
impersonationResourceNames: ["some-group"]

Expected behavior After adding rbac configuration block impersonate limitation works and in UI we can see cluster data.

Actual Behavior After adding rbac configuration block with specific group in UI there is no cluster data.

LukaszRacon commented 2 years ago

Two issues:

impersonation for groups requires access to impersonate on users resource at cluster scope level (overall bad idea)
impersonationResourceNames: ["gitops-reader"] does not match group in logs "groups": ["wego-admin-rk"]

Following config requires access to impersonate on users resource at cluster scope level

rbac:
  create: true
  impersonationResources: ["groups"]
  impersonationResourceNames: ["gitops-reader"]

With current configuration it fails with error: {"cluster": "Default", "user": "raman", "error": "user namespace access: users \"raman\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops-weave-gitops\" cannot impersonate resource \"users\" in API group \"\" at the cluster scope"}

Here is a fix (adjust names/namespaces accordingly)

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ww-gitops-weave-gitops-patch
rules:
  - apiGroups:
      - ""
    resources:
      - users
    verbs:
      - impersonate
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ww-gitops-weave-gitops-patch
subjects:
  - kind: ServiceAccount
    name: ww-gitops-weave-gitops
    namespace: flux-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ww-gitops-weave-gitops-patch

grglzrv commented 1 year ago

I have the same issue. Configured Okta successfully with Weave server but not seeing any data.

Tried the aforementioned solutions but without success

i am getting this error

 2022-12-19T16:39:53.985Z    ERROR    gitops.core-server    server/version.go:105    error getting flux version    {"error": "error getting list of objects"}                                                                             │
│ 2022-12-19T16:39:55.200Z    INFO    gitops.auth-server    auth/jwt.go:77    attempt to read token from auth header                                                                                                                       │
│ 2022-12-19T16:39:55.201Z    INFO    gitops.auth-server    auth/jwt.go:77    attempt to read token from auth header

and "No data" into Weave UI

versions: Kubernetes: v1.24.7-eks-fb459a0 Flux: 0.33 Weave GitOps: v0.12.0

rbac config:

    rbac:
      create: true
      impersonationResourceNames: ["admin", "AdminGroup"]
      impersonationResources: ["users", "groups"]
      viewSecretsResourceNames: ["cluster-user-auth", "oidc-auth"]

cluster role and binding

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: wego-admin-cluster-role-okta
rules:
  - apiGroups: [""]
    resources: ["secrets", "pods" ]
    verbs: [ "get", "list" ]
  - apiGroups: ["apps"]
    resources: [ "deployments", "replicasets"]
    verbs: [ "get", "list" ]
  - apiGroups: ["kustomize.toolkit.fluxcd.io"]
    resources: [ "kustomizations" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: ["helm.toolkit.fluxcd.io"]
    resources: [ "helmreleases" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: ["source.toolkit.fluxcd.io"]
    resources: [ "buckets", "helmcharts", "gitrepositories", "helmrepositories", "ocirepositories" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: wego-cluster-admin-okta
subjects:
- kind: Group
  name: AdminGroup
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: wego-admin-cluster-role-okta
  apiGroup: rbac.authorization.k8s.io

also added and this "patch" cluster role and binding

  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: flux-webui-weave-gitops-patch
  rules:
    - apiGroups:
        - ""
      resources:
        - users
        - groups
      verbs:
        - impersonate
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: flux-webui-weave-gitops-patch
  subjects:
    - kind: ServiceAccount
      name: flux-webui-weave-gitops
      namespace: flux-system
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: flux-webui-weave-gitops-patch

@JamWils do you know if anyone is working on it ? @LukaszRacon did you tested it with K8s version 1.24?

Bare in mind that in k8s version the SA flux-webui-weave-gitops does not have token cuz the token has to be created manually

│ Name:                flux-webui-weave-gitops                                                                                                                                                                                             │
│ Namespace:           flux-system                                                                                                                                                                                                         │
│ Labels:              app.kubernetes.io/instance=flux-webui                                                                                                                                                                               │
│                      app.kubernetes.io/managed-by=Helm                                                                                                                                                                                   │
│                      app.kubernetes.io/name=weave-gitops                                                                                                                                                                                 │
│                      app.kubernetes.io/version=v0.12.0                                                                                                                                                                                   │
│                      helm.sh/chart=weave-gitops-4.0.9                                                                                                                                                                                    │
│                      helm.toolkit.fluxcd.io/name=flux-webui                                                                                                                                                                              │
│                      helm.toolkit.fluxcd.io/namespace=flux-system                                                                                                                                                                        │
│ Annotations:         meta.helm.sh/release-name: flux-webui                                                                                                                                                                               │
│                      meta.helm.sh/release-namespace: flux-system                                                                                                                                                                         │
│ Image pull secrets:  <none>                                                                                                                                                                                                              │
│ Mountable secrets:   <none>                                                                                                                                                                                                              │
│ Tokens:              flux-webui-weave-gitops-token                                                                                                                                                                                       │
│ Events:              <none>                                                                                                                                                                                                              │
│

for example:

---
apiVersion: v1
kind: Secret
metadata:
  name: flux-webui-weave-gitops-token 
  namespace: flux-system
  annotations:
    kubernetes.io/service-account.name: "flux-webui-weave-gitops"
type: kubernetes.io/flux-webui-weave-gitops-token

LukaszRacon commented 1 year ago

I am running a custom build that includes PR #2745 - I am using AzureAD with AKS 1.24.6

grglzrv commented 1 year ago

@JamWils do you know if there is any update from team/pesto on this matter?

mikalai-t commented 1 year ago

Hello. The same issue here - no data in UI when trying to restrict access based on groups in Azure (it shouldn't really matter as OAuth2/OIDC is more or less similarly implemented in any Identity Provider). I had applied all the necessary fixes:

additional ClusterRole and ClusterRoleBinding to impersonate all the users - as suggested by @grglzrv

added cli not to request groups scope as AzureAD doesn't support it:

additionalArgs:
  - --auth-methods=oidc
  - --oidc-client-id={{ .clientID }}
  - --oidc-client-secret={{ .clientSecret }}
  - --oidc-redirect-url={{ .redirectURL }}
  - --oidc-issuer-url={{ .issuerURL }}
  - --custom-oidc-scopes=openid,profile,email

but ended up with the kind of filtering error when the app throws an error on the first group and stops processing the rest, e.g. my user belongs to 5 groups:

2023-02-13T20:44:05.587Z    DEBUG   gitops  middleware/middleware.go:57 request success {"uri": "/v1/objects", "status": 200}
2023-02-13T20:44:06.213Z    DEBUG   gitops  middleware/middleware.go:57 request success {"uri": "/", "status": 200}
2023-02-13T20:44:06.213Z    DEBUG   gitops  middleware/middleware.go:57 request success {"uri": "/", "status": 200}
2023-02-13T20:44:10.690Z    INFO    gitops.auth-server  auth/jwt.go:77  attempt to read token from auth header
2023-02-13T20:44:10.690Z    DEBUG   gitops.auth-server  auth/jwt.go:54  parsing cookie JWT token    {"claimsConfig": {"Username":"email","Groups":"groups"}}
2023-02-13T20:44:10.690Z    DEBUG   gitops.auth-server  auth/jwt.go:161 Found principal {"user": "FirstLastname@example.com", "groups": ["1111-0000-2222-...", "3333-0000-4444-...", "5555-MyTarget-Group-UUID" ], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2023-02-13T20:44:10.988Z    INFO    gitops.auth-server  auth/jwt.go:77  attempt to read token from auth header
2023-02-13T20:44:10.988Z    DEBUG   gitops.auth-server  auth/jwt.go:54  parsing cookie JWT token    {"claimsConfig": {"Username":"email","Groups":"groups"}}
2023-02-13T20:44:10.989Z    DEBUG   gitops.auth-server  auth/jwt.go:161 Found principal ...omitted for brevity
2023-02-13T20:44:11.089Z    ERROR   gitops  clustersmngr/factory.go:566 failed filtering namespaces {"cluster": "Default", "user": "FirstLastname@example.com", "error": "user namespace access: groups \"1111-0000-2222-...\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops\" cannot impersonate resource \"groups\" in API group \"\" at the cluster scope"}

Also trying to filter by user throws exactly the same error (and a work around is to create another extra ClusterRole again), so I would say a way to build a final RBAC manifest from Helm chart could be improved.

aboutobza-talend commented 1 year ago

@grglzrv did you find any solution for the issue ?

grglzrv commented 1 year ago

@grglzrv did you find any solution for the issue ?

Unfortunately no, I gonna try again this week.

grglzrv commented 1 year ago

@aboutobza-talend you may try this HelmRelease - oidcSecret : https://github.com/FrankelJb/home-ops/blob/main/kubernetes/apps/flux-system/weave-gitops/helmrelease.yaml#L34-L39 and impersonationResourceNames: https://github.com/FrankelJb/home-ops/blob/main/kubernetes/apps/flux-system/weave-gitops/helmrelease.yaml#L60-L62

Cajga commented 12 months ago

additional ClusterRole and ClusterRoleBinding to impersonate all the users - as suggested by @grglzrv
added cli not to request groups scope as AzureAD doesn't support it:
additionalArgs:
- --auth-methods=oidc
- --oidc-client-id={{ .clientID }}
- --oidc-client-secret={{ .clientSecret }}
- --oidc-redirect-url={{ .redirectURL }}
- --oidc-issuer-url={{ .issuerURL }}
- --custom-oidc-scopes=openid,profile,email

AzureAD does support groups. Go to the app registration -> Token configuration -> Add groups claim

weaveworks / weave-gitops

Okta group impersonate limitation #2696