GitLab Groups are not fetched from userinfo endpoint #5944

Open MatteoJoliveau opened 3 years ago

Checklist:

[x] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
[x] I've included steps to reproduce the bug.
[x] I've pasted the output of argocd version.

Describe the bug

When using GitLab as an OIDC provider, groups are not mapped to Argo and are therefore not usable in RBAC policies. GitLab does not return group membership info in the ID token, only in the /userinfo endpoint. I suspect ArgoCD is not calling that endpoint for user information but only reading the ID token.

To Reproduce

Configure GitLab as an OIDC provider with the following configuration:

apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cm
namespace: argocd
data:
oidc.config: |
name: GitLab
issuer: https://gitlab.example.com
clientID: REDACTED
clientSecret: $oidc.gitlab.clientSecret
# the groups scope is not supported by GitLab. Groups are returned as part of the openid scope
requestedScopes: [openid profile email]

check https://argocd.example.com/user-info for the list of groups
confirm that the GitLab groups are NOT showing in the Groups section

Expected behavior

ArgoCD calls the /userinfo endpoint of the GitLab instance to retrieve group membership info.

Screenshots

Version

> argocd version
argocd: v1.4.2+48cced9
  BuildDate: unknown
  GitCommit: 48cced9d925b5bc94f6aa9fa4a8a19b2a59e128a
  GitTreeState: clean
  GoVersion: go1.15.10
  Compiler: gc
  Platform: linux/amd64
argocd-server: v2.0.0-rc1+0ca643f
  BuildDate: 2021-03-19T21:27:59Z
  GitCommit: 0ca643f027b99e8a5b931bb8ee9df42c3e4b64bf
  GitTreeState: clean
  GoVersion: go1.16
  Compiler: gc
  Platform: linux/amd64
  Ksonnet Version: v0.13.1
  Kustomize Version: v3.9.4 2021-02-09T19:22:10Z
  Helm Version: v3.5.1+g32c2223
  Kubectl Version: v0.20.4

Logs

Notice the lack of groups claim in the field grpc.request.claims.

time="2021-04-01T12:21:44Z" level=info msg="received unary call /session.SessionService/GetUserInfo" grpc.method=GetUserInfo grpc.request.claims="{\"aud\":\"353ab692234207c659960bcbc31696753ee13f6502e3f32c03d49b682ab00221\",\"auth_time\":1617275574,\"email\":\"matteo.joliveau@mikamai.com\",\"email_verified\":true,\"exp\":1617279821,\"iat\":1617279701,\"iss\":\"https://gitlab.example.com\",\"sub\":\"87\",\"sub_legacy\":\"718eb3690efda2cdebed7e194436a5405b2a1e55854aa4403a7437acd0c6d158\"}" grpc.request.content= grpc.service=session.SessionService grpc.start_time="2021-04-01T12:21:44Z" span.kind=server system=grpc

I wonder if the Gitlab group membership endpoint needs to be ..

called in the dex connector code and *have the token regenerated with the group membership information

.. before returning it?

On Thu, Apr 1, 2021, 08:39 Matteo Joliveau @.***> wrote:

Checklist:

I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.

I've included steps to reproduce the bug.

I've pasted the output of argocd version.

Describe the bug

When using GitLab https://docs.gitlab.com/ee/integration/openid_connect_provider.html as an OIDC provider, groups are not mapped to Argo and are therefore not usable in RBAC policies. GitLab does not return group membership info in the ID token, only in the /userinfo endpoint. I suspect ArgoCD is not calling that endpoint for user information but only reading the ID token.

To Reproduce

Configure GitLab as an OIDC provider with the following configuration:

apiVersion: v1kind: ConfigMapmetadata: name: argocd-cm namespace: argocddata: oidc.config: | name: GitLab issuer: https://gitlab.example.com clientID: REDACTED clientSecret: $oidc.gitlab.clientSecret # the groups scope is not supported by GitLab. Groups are returned as part of the openid scope requestedScopes: [openid profile email]

check https://argocd.example.com/user-info for the list of groups

confirm that the GitLab groups are NOT showing in the Groups section

Expected behavior

ArgoCD calls the /userinfo endpoint of the GitLab instance to retrieve group membership info.

Screenshots

[image: image] https://user-images.githubusercontent.com/9077612/113294645-8e7bfd80-92f7-11eb-8343-97014871823c.png

Version

argocd version argocd: v1.4.2+48cced9 BuildDate: unknown GitCommit: 48cced9d925b5bc94f6aa9fa4a8a19b2a59e128a GitTreeState: clean GoVersion: go1.15.10 Compiler: gc Platform: linux/amd64 argocd-server: v2.0.0-rc1+0ca643f BuildDate: 2021-03-19T21:27:59Z GitCommit: 0ca643f027b99e8a5b931bb8ee9df42c3e4b64bf GitTreeState: clean GoVersion: go1.16 Compiler: gc Platform: linux/amd64 Ksonnet Version: v0.13.1 Kustomize Version: v3.9.4 2021-02-09T19:22:10Z Helm Version: v3.5.1+g32c2223 Kubectl Version: v0.20.4

Logs

Notice the lack of groups claim in the field grpc.request.claims.

time="2021-04-01T12:21:44Z" level=info msg="received unary call /session.SessionService/GetUserInfo" grpc.method=GetUserInfo @.***\",\"email_verified\":true,\"exp\":1617279821,\"iat\":1617279701,\"iss\":\"https://gitlab.example.com\",\"sub\":\"87\",\"sub_legacy\":\"718eb3690efda2cdebed7e194436a5405b2a1e55854aa4403a7437acd0c6d158\"}" grpc.request.content= grpc.service=session.SessionService grpc.start_time="2021-04-01T12:21:44Z" span.kind=server system=grpc

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/argoproj/argo-cd/issues/5944, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFEAEARRH3Y2V2FFFBGHLTGRSQ7ANCNFSM42HATRXQ .

Just as a side note, we currently can't use Dex because our clusters have a dedicated Dex instance (i.e. not the one bundled with Argo, but a standalone one managed by the platform), at least until Argo can support reading secrets other than argocd-secrets. Our Dex Operator generates and rotates its own secrets, so it isn't compatible with Argo at the moment.

We're waiting for #4342 to be merged to integrate with Dex, but until then we're stuck with direct SSO config.

Besides that, I think that if ArgoCD officially supports OIDC as an auth mechanism, it should work with a popular provider such as GitLab without requiring an additional tool like Dex. In case this is not the case, it should be explicitly noted on the docs that GitLab/GitLab groups are unsupported

Hi @MatteoJoliveau! I see what you are saying. Makes sense. At the moment, ArgoCD supports jwt but doesn't do much with the authentication handshake. That is pretty much delegated to an external broker like dex.

Re: Dex & Gitlab ( you are probably already aware, wanted to note this down ) Groups are supported here https://github.com/dexidp/dex/blob/master/connector/gitlab/gitlab.go#L161

4342 seems to be pretty old, we'll check why it didn't make it.

Hi @sbose78!

Yes I'm aware that GitLab groups work with Dex. We are indeed using them for other services that authenticate against Dex (like Grafana). But as I described in my previous comment, until we have a way to inject the OIDC credentials into ArgoCD from another secret we can't use this strategy.

@sbose78 sorry to ping, but any news on this one?

I was thinking about the best way of fixing this, and so far I think we only have two options:

edit ArgoCD's code for OpenID Connect so that it fetches claims from the standard OAuth UserInfo endpoint in addition/instead of the ID Token.
edit ArgoCD's code for secrets handling so that it's able to read values from the environment and/or from secrets other than argocd-secret, so that we can plug our own Dex instance in and let it handle group mapping. In that case we should also add a note in the documentation about the fact that GitLab groups (and possibly other providers exhibiting the same behaviour) are not supported when using OIDC directly, but only with Dex.

I think the latter is a reasonable approach in the short term, though I'm for solving it better than that. There's already wide consensus on #4342 .

@alexmt what do you think? Also, Matteo, would you be up for joining the contributors office hours to present your problem and drive a discussion on the proposed solutions?

@sbose78 I would love to if I manage to have time available. When are office hours usually held?

@MatteoJoliveau Have you tried adding getUserInfo: true to your OIDC config (same level as clientID and clientSecret)? I've had to do the same when using Okta as an OIDC source to get group claims to come through on dex.

You might additionally also have to enable insecureEnableGroups: true. Per dex docs:

    # Groups claims (like the rest of oidc claims through dex) only refresh when the id token is refreshed
    # meaning the regular refresh flow doesn't update the groups claim. As such by default the oidc connector
    # doesn't allow groups claims. If you are okay with having potentially stale group claims you can use
    # this option to enable groups claims through the oidc connector on a per-connector basis.
    # This can be overridden with the below option
    # insecureEnableGroups: true

    # When enabled, the OpenID Connector will query the UserInfo endpoint for additional claims. UserInfo claims
    # take priority over claims returned by the IDToken. This option should be used when the IDToken doesn't contain
    # all the claims requested.
    # https://openid.net/specs/openid-connect-core-1_0.html#UserInfo
    # getUserInfo: true

@moensch our Dex instance works, the problem is that as I stated above we manage our own Dex instance using an operator. Said operator writes the client credentials to a dedicated Kubernetes secret. At the moment ArgoCD is not able to read from multiple secrets, therefore we cannot use Dex with ArgoCD for now.

Matteo, Thursday 12noon Eastern time :)

On Tue, Apr 20, 2021 at 4:40 AM Matteo Joliveau @.***> wrote:

@moensch https://github.com/moensch our Dex instance works, the problem is that as I stated above we manage our own Dex instance using an operator. Said operator writes the client credentials to a dedicated Kubernetes secret. At the moment ArgoCD is not able to read from multiple secrets, therefore we cannot use Dex with ArgoCD for now.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/argoproj/argo-cd/issues/5944#issuecomment-823095234, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEFEABROND4OSSMNHYP33TTJU4Y5ANCNFSM42HATRXQ .

Here to say I'm getting the same symptoms with Okta as OIDC provider even though I did everything according to: https://argoproj.github.io/argo-cd/operator-manual/user-management/okta/ (the without dex implementation)

Same as @MatteoJoliveau, I'm not seeing the groups claim ever requested in the logs. I even tried @moensch 's suggestion and to no avail. Anyone managed to get it to work with Okta?

@asafhm: Okta requires passing custom claims (assuming you haven't configured that already). I've had success using Dex instead of the built in OIDC integration using Dex and enabling both getUserInfo and insecureEnableGroups. The group information is good until the token is reissued so if stale group information is a problem you can mitigate it by reducing the lifetime of the token.

@sbose78 Would adding documentation for using Dex with OIDC on the https://argoproj.github.io/argo-cd/operator-manual/user-management/ page be useful?

Absolutely, please! 🙇

Hi, I have the same problems and for me configuring using dex does not work. I am sure we do not have any dex instance running in our environment which might cause conflict. If I use dex then after clicking the auth button it just refresh the browser quickly and goes back to where it was without any error or more useful information.

However using direct OIDC seems to work at least I can authenticate and login. But the scope can not have "groups" (yet gitlab oidc does not offer that) but we need the ability to fetch group and mapp to argocd group so we can manage RBAC. Stuck here :P

Hi, when configuring user authentication with direct OIDC with Gitlab I can also connect but I don't have any group. This is normal because according to Gitlab's documentation,

The claims sub, sub_legacy, email, email_verified and groups_direct are included in the ID token. All other claims are available from the /oauth/userinfo endpoint used by OIDC clients.

And indeed, I can see my direct groups in the groups_direct claim when I'm decoding my JWT.

I'm wondering if there is a way to use this claim for group mapping?

Actually I found a solution: just put this in argocd-rbac-cm: scopes: '[groups_direct]'

I think using scopes: '[groups_direct]' is a workaround. The groups_direct claim does not include the GitLab groups where a user is included via a group of users.

The groups claim from userinfo endpoint should return all group the user is member off -> Gitlab docs.

The config for userinfo is documented here: Retrieving group claims when not in the token in your case it would be:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  oidc.config: |
    name: GitLab
    issuer: https://gitlab.example.com
    clientID: REDACTED
    clientSecret: $oidc.gitlab.clientSecret
    # the groups scope is not supported by GitLab. Groups are returned as part of the openid scope
    requestedScopes: [openid profile email]
    enableUserInfoGroups: true
    userInfoPath: /oauth/userinfo
    userInfoCacheExpiration: "5m"

argoproj / argo-cd

GitLab Groups are not fetched from userinfo endpoint #5944

4342 seems to be pretty old, we'll check why it didn't make it.