Logging in with OIDC still redirects me to login page

schen2315 commented 3 years ago

Hey guys, I setup oidc and after authenticating am brought back to the login page (login with xxxx) and can't access the actual application. When i go to /api/me i see logged is set to true, my username is there and the roles that my company org returned. Do the roles returned by my provider have to match exactly to the roles used by akhq (e.g. topic/read, topic/insert, etc. ). I am on 0.16.0

Here's my config:

micronaut:
  security:
    enabled: true
    oauth2:
      enabled: true
      clients:
        bsso-oauth2:
          client-id: <MY_CLIENT_ID>
          client-secret: <MY_CLIENT_SECRET>
          openid:
            issuer: <MY_PROVIDER>
security:
  default-group: no-roles
  oidc:
    enabled: true
    providers:
      bsso-oauth2:
        label: "Login with MY_PROVIDER"
        username-field: preferred_username
        groups-field: roles
        default-group: reader

tchiotludo commented 3 years ago

Is this the full configuration ? I see that you use default group and there is an issue https://github.com/tchiotludo/akhq/pull/526 that can impact you ?

schen2315 commented 3 years ago

i only put the parts that i thought were relevant, only other part of my config is my kafka connections. Ill take a look at #526

schen2315 commented 3 years ago

i updated my config to use the suggestion made in #526 :

security:
  default-group: no-roles
  groups:
  - name: admin
    roles:
    - topic/read
    - topic/insert
    - topic/delete
    - topic/config/update
    - node/read
    - node/config/update
    - topic/data/read
    - topic/data/insert
    - topic/data/delete
    - group/read
    - group/delete
    - group/offsets/update
    - registry/read
    - registry/insert
    - registry/update
    - registry/delete
    - registry/version/delete
    - acls/read
    - connect/read
    - connect/insert
    - connect/update
    - connect/delete
    - connect/state/update
  - name: reader
    roles:
    - topic/read
    - node/read
    - topic/data/read
    - group/read
    - registry/read
    - acls/read
    - connect/read
  - name: no-roles
    roles: []
  oidc:
    enabled: true
    providers:
      oauth2:
        label: "Login with MY_PROVIDER"
        username-field: preferred_username
        groups-field: roles
        groups:
          - name: ROLE_RETURNED_BY_PROVIDER
            groups:
              - reader

However, after logging in i still get redirected to the login page (login with xxxx). When i go to /api/me i get something like:

{"logged":true,"username":"schen2315","roles":["ROLE_1", "ROLE_2", "ROLE_3"]}

Is there something wrong with my config?

tchiotludo commented 3 years ago

Not really sure to have understand ? Are you using role from akhq group ? or role coming from OIDC provider ?

Seems to be confusing here !

Just see also that you mix akhq & micronaut configuration.

akhq: 
  security:
    default-group: no-roles
    groups:

default & group, groups, .... are akhq configuration, not the micronaut one.

schen2315 commented 3 years ago

my bad, im pasting what i think are relevant parts. My config does have security under akhq. This is what my full config looks like:

    micronaut:
      security:
        enabled: true
        oauth2:
          enabled: true
          clients:
            oauth2:
              client-id: MY_CLIENT_ID
              client-secret: MY_CLIENT_SECRET
              openid:
                issuer: MY_PROVIDER
    akhq:
      connections:
        kafkasub:
          properties:
            bootstrap.servers: <MY_KAFKA_BROKER>
      topic-data:
        poll-timeout: 10000
      security:
        default-group: no-roles
        groups:
        - name: admin
          roles:
          - topic/read
          - topic/insert
          - topic/delete
          - topic/config/update
          - node/read
          - node/config/update
          - topic/data/read
          - topic/data/insert
          - topic/data/delete
          - group/read
          - group/delete
          - group/offsets/update
          - registry/read
          - registry/insert
          - registry/update
          - registry/delete
          - registry/version/delete
          - acls/read
          - connect/read
          - connect/insert
          - connect/update
          - connect/delete
          - connect/state/update
        - name: reader
          roles:
          - topic/read
          - node/read
          - topic/data/read
          - group/read
          - registry/read
          - acls/read
          - connect/read
        - name: no-roles
          roles: []
        oidc:
          enabled: true
          providers:
            oauth2:
              label: "Login with MY_PROVIDER"
              username-field: sub
              groups-field: roles
              groups:
                - name: ROLE_1
                  groups:
                    - reader

Im using the role coming from my oidc provider. My understanding is that

              groups:
                - name: ROLE_1
                  groups:
                    - reader

means assign a user whose role ROLE_1 coming from the oidc provider is given the group reader coming from akhq

tchiotludo commented 3 years ago

to be honest, I never try this part, since I don't have any custom OIDC provider, i only try standard openid, @jniebuhr do it and maybe he have more information about this ? As I understand, the OIDC custom provider must return group and not role to be working. Maybe you can try ?

schen2315 commented 3 years ago

so we should set our oidc provider to return something like:

{"logged":true,"username":"schen2315","groups":["ROLE_1", "ROLE_2", "ROLE_3"]}

i believe for ldap, you are able to map groups returned by the ldap server to custom groups created in akhq. Is it not the same for oidc?

tchiotludo commented 3 years ago

As I understand here : https://github.com/tchiotludo/akhq/blob/5ab788da22529ede3c7b640f1918304511cea6f0/src/main/java/org/akhq/modules/OidcUserDetailsMapper.java#L106

The default return was more like that :

{"logged":true,"username":"schen2315","roles":["AKHQGROUP1", "AKHQGROUP2", "AKHQGROUP2"]}

Not really sure about that here, sorry ! We need more docs about that ;)

jniebuhr commented 3 years ago

@schen2315 You are correct, OIDC contains the same logic as LDAP. So the mapping is OIDC Groups -> AKHQ Groups -> AKHQ Roles. So it should work. There might be a conflict if the OIDC Claim is called roles. But I thought I put something in there to solve that.

hoeggi commented 3 years ago

I'm working on this exact same issue right now too. I found this comment: https://github.com/tchiotludo/akhq/issues/485#issuecomment-719603022 saying that using the role name instead of a group should work. Can anyone confirm this for oidc? I'm unable to check this myself because of a bug in keycloak that prevents roles with a / in the name.

I will try to change the claim field to something else form the default role but not sure yet if this is possible in keycloak.

My config looks pretty much the same as the one from @schen2315 except that i also have a jwt token in my config:

        micronaut:
          security:
            enabled: true
            oauth2:
              enabled: true
              clients:
                oidc:
                  client-id: "akhq-ui"
                  client-secret: "XXX-XXX-XXX"
                  openid:
                    issuer: "https://XXX-XXX-XXX/auth/realms/infrastructure/"
            token:
              jwt:
                signatures:
                  secret:
                    generator:
                      secret: XXX-XXX-XXX

Happy to also provide more of my config or some logs if necessary

hoeggi commented 3 years ago

I tried to change the name of the field in my claim from roles to akhqgroup and changed the groups-field to the new name. This prevents akhq from starting whith the following exception:

Error starting Micronaut server: Failed to inject value for parameter [groups] of method [setGroups] of class: org.akhq.configs.SecurityProperties
Message: Error resolving property value [akhq.security.groups]. Unable to convert value [[{name=oidc-admin, roles=[topic/read, topic/insert, topic/delete, topic/config/update, node/read, node/config/update, topic/data/read, topic/data/insert, topic/data/delete, group/read, group/delete, group/offsets/update, registry/read, registry/insert, registry/update, registry/delete, registry/version/delete, acls/read, connect/read, connect/insert, connect/update, connect/delete, connect/state/update]}, {name=oidc-reader, roles=[topic/read, node/read, topic/data/read, group/read, registry/read, acls/read, connect/read]}, {name=no-roles, roles=[]}]] to target type [Map<String, Group>] due to: Cannot convert an iterable with more than 1 value to a non collection object
Path Taken: DefaultOpenIdClient.openIdClient(OpenIdClientConfiguration openIdClientConfiguration,OauthClientConfiguration clientConfiguration,Provider openIdProviderMetadata,OpenIdUserDetailsMapper userDetailsMapper,AuthorizationRedirectHandler redirectUrlBuilder,[OpenIdAuthorizationResponseHandler authorizationResponseHandler],EndSessionEndpointResolver endSessionEndpointResolver,EndSessionCallbackUrlBuilder endSessionCallbackUrlBuilder) 
--> new DefaultOpenIdAuthorizationResponseHandler(OpenIdTokenResponseValidator tokenResponseValidator,[DefaultOpenIdUserDetailsMapper userDetailsMapper],TokenEndpointClient tokenEndpointClient,OauthRouteUrlBuilder oauthRouteUrlBuilder,StateValidator stateValidator) 
--> new OidcUserDetailsMapper(OpenIdAdditionalClaimsConfiguration openIdAdditionalClaimsConfiguration,AuthenticationModeConfiguration authenticationModeConfiguration,Oidc oidc,[UserGroupUtils userGroupUtils]) 
--> UserGroupUtils.securityProperties 
--> SecurityProperties.setGroups([Map groups])

io.micronaut.context.exceptions.DependencyInjectionException: Failed to inject value for parameter [groups] of method [setGroups] of class: org.akhq.configs.SecurityProperties
Message: Error resolving property value [akhq.security.groups]. Unable to convert value [[{name=oidc-admin, roles=[topic/read, topic/insert, topic/delete, topic/config/update, node/read, node/config/update, topic/data/read, topic/data/insert, topic/data/delete, group/read, group/delete, group/offsets/update, registry/read, registry/insert, registry/update, registry/delete, registry/version/delete, acls/read, connect/read, connect/insert, connect/update, connect/delete, connect/state/update]}, {name=oidc-reader, roles=[topic/read, node/read, topic/data/read, group/read, registry/read, acls/read, connect/read]}, {name=no-roles, roles=[]}]] to target type [Map<String, Group>] due to: Cannot convert an iterable with more than 1 value to a non collection object
Path Taken: DefaultOpenIdClient.openIdClient(OpenIdClientConfiguration openIdClientConfiguration,OauthClientConfiguration clientConfiguration,Provider openIdProviderMetadata,OpenIdUserDetailsMapper userDetailsMapper,AuthorizationRedirectHandler redirectUrlBuilder,[OpenIdAuthorizationResponseHandler authorizationResponseHandler],EndSessionEndpointResolver endSessionEndpointResolver,EndSessionCallbackUrlBuilder endSessionCallbackUrlBuilder) --> new DefaultOpenIdAuthorizationResponseHandler(OpenIdTokenResponseValidator tokenResponseValidator,[DefaultOpenIdUserDetailsMapper userDetailsMapper],TokenEndpointClient tokenEndpointClient,OauthRouteUrlBuilder oauthRouteUrlBuilder,StateValidator stateValidator) 
--> new OidcUserDetailsMapper(OpenIdAdditionalClaimsConfiguration openIdAdditionalClaimsConfiguration,AuthenticationModeConfiguration authenticationModeConfiguration,Oidc oidc,[UserGroupUtils userGroupUtils]) 
--> UserGroupUtils.securityProperties 
--> SecurityProperties.setGroups([Map groups])

I'm also not 100% what the groups-field really means (something in the akhq config or the name inside the claim?). For reference, here is part of the claim:

{
  "sub": "XXXX",
  "email_verified": true,
  "akhqgroup": [
    "topic-writer",
    "admin"
  ],
  "roles": [],
  "iss": "akhq",
  "typ": "ID",
  "preferred_username": "XXXX",
  "locale": "de",
  "given_name": "XXXX",
  "nonce": "XXXX",
  "acr": "1",
  "nbf": 1607520445,
  "azp": "akhq-ui",
  "auth_time": 1607520444,
  "name": "XXXX",
  "exp": 1607524045,
  "session_state": "1838aebc-225f-4972-b45e-2bf3227b7e30",
  "iat": 1607520445,
  "family_name": "XXXX",
  "email": "XXXX",
  "oauth2Provider": "oidc"
}

I tried the same with the role in the role field of the json and with only one entry instead of 2.

tchiotludo commented 3 years ago

As I understand your payload must be :

{
  "sub": "XXXX",

  "roles": [
    "topic-writer",
    "admin"
  ],
  "iss": "akhq",

}

do you try with that ? For now, don't add special others akhq group to try

hoeggi commented 3 years ago

I tried with that payload, and had the same redirect problem. I checkt:

a topic-writer custom group
admin without specifiing anything, since this works as a build in group when I use basic auth What I could not try was adding a role instead of a group (eg.: registry/delete or topic/read since a bug in keycloak prevents me from creating roles with a / in the name.

Could you point in at the code where this mapping happens, I couldn't find it but I might be able to debug into this a little bit.

tchiotludo commented 3 years ago

The whole logic is here : https://github.com/tchiotludo/akhq/blob/5ab788da22529ede3c7b640f1918304511cea6f0/src/main/java/org/akhq/modules/OidcUserDetailsMapper.java#L72 I think we could detail how it work and maybe adjust some naming that lead to confusion and make a big doc about that !

unfortunately, i don't have a custom oidc provider myself

hoeggi commented 3 years ago

I finally tracked down the problem. When the UserDetails are build with createUserDetails in OidcUserDetailsMapper it calls buildAttributes. The Problem with this is that the attributes then contain the roles field from the claim. In my case this is admin since i wanted to use the default groups already in akhq. But since there is no admin role it fails because it's impossible to assign a role to the user.

This is also the reason why it works if your OIDC provider puts roles instead of groups into the claim.

Question now is, what should be the intended behavior here? My personal preference would be to provide group memberships and not roles directly since it would mean creating all roles in your OIDC provider instead of just some groups that wrap all intended roles on akhq side (also if I read the doc correctly I think this is how it's supposed to work?) but not sure how other users use this.

hoeggi commented 3 years ago

This is how I got it working https://github.com/hoeggi/akhq/commit/341bc0d9ca6e4e0c2065ec406d46ec4eacd3c45b depending on what the actual intended behavior is I can open this as a PR if you want me to

tchiotludo commented 3 years ago

@hoeggi I would also prefer group from akhq, I think it was the actual behaviour.

Can you provide a PR on that?
Can you refactor the class (there is a lot of roles var and it's will not be the case)?
Maybe you can also add some documentation about that on README?

Thanks for that

hoeggi commented 3 years ago

I'll update the README and open a PR. Can you elaborate a bit more on the refactoring, not sure what exactly you mean here.

tchiotludo commented 3 years ago

for example:

        Oidc.Provider provider = oidc.getProvider(providerName);
        if (attributes.containsKey(provider.getGroupsField())) {
            attributes.put(provider.getGroupsField(), roles);
        }

getGroupsFields, roles seems to be a bug at first look :smile:

hoeggi commented 3 years ago

It's actually the opposite :) without this, the attributes contain a field:

roles=["admin"]

what this does is, it overrides the value of the roles key so you get this instead (with all roles that match previously):

roles=["topic/reader","topic/writer" ... ]

In this example the roles is what you configure in your application.yaml as groups-field I didn't go deep enough into micronaut to find out how it maps the roles but you can compare it if you check /api/me

tchiotludo commented 3 years ago

So you want to provide roles from oidc instead of group ? In this case, add a configuration parameter getRolesFields I think

archetec commented 3 years ago

Hello. We are experiencing the same problem with OIDC using Azure AD. It looks like we are logged in (we can see the usernames in the logs) but we are stuck at the login page and can't access anything in AKHQ. It's difficult to investigate since there is no DEBUG log in AKHQ so... we think it's a problem in the role and AKHQ group mapping code, but can't confirm. Is there a solution?

hoeggi commented 3 years ago

Took me a while to get back to this,

So you want to provide roles from oidc instead of group ? In this case, add a configuration parameter getRolesFields I think

So the problem is this:

if you use basic auth and have an akhq group assigned to the user the group is correctly mapped to the roles associated with the group and the roles field contains the correct roles, like this:
```
roles=["topic/reader","topic/writer" ... ]
```
if you use oidc to login and have an akhq group assigned to the user the roles are not correctly set and the roles field contains the group the user is assigned to instead of the roles, like this:
```
roles=["admin"]
```
what i did here:
```
Oidc.Provider provider = oidc.getProvider(providerName);
if (attributes.containsKey(provider.getGroupsField())) {
attributes.put(provider.getGroupsField(), roles);
}
```
was to override the wrongly mapped roles=["admin"] with the correct roles=["topic/reader","topic/writer" ... ]

tchiotludo commented 3 years ago

@hoeggi maybe a PR for that with some docs ? Seems that there is a misunderstanding on that and it will help !

Thanks

tchiotludo / akhq

Logging in with OIDC still redirects me to login page #534