Keycloak/Quarkus Issues: Dev and Prod

tmulle commented 1 year ago

Describe the bug

So I notice a few things with the Keycloak application itself, both in dev mode and prod.

Dev Mode:

The Swagger UI won't let me log in with "client_credentials" I get a "Failed to load" error after entering my client-id and secrect.

TypeError: Load failed

Random times I'd get the Keycloak page saying "Invalid redirect_uri" when trying to log using the Swagger UI clicking on the Authorize button.

Import existing Realm:

The client secret is not imported when importing an existing realm. This causes me to have to regenerate the secret and update all running applications in production. This is not good when we deploy a new server using our realm configuration.

The realms are created from the same version of Quarkus Keycloak. I've tried both 21.x and 22.x and the same thing. The Dev Services in Quarkus 3.3.0 shows the same behavior.

I saw this post on Keycloak Github and wondering if this is by design or a bug? It effects the Dev Service as well as it doesn't import my client secrets It just makes the secret '**'

https://github.com/keycloak/keycloak/issues/9201

Expected behavior

I should be able to log into Keycloak and also have all my information imported from an existing realm.

Actual behavior

No response

How to Reproduce?

Import issue:

Create a realm on Quarkus Keycloak (21.x, 22.x)
Create a client with a secret
Export the realm
Either use Dev Services or Standalone Keycloak to import the exported
Look in the Clients -> <Your client> -> Credentials tab and notice the secret field contains "***" and not your secret.

Dev UI/ Swagger:

Try to log into your keycloak instance using client_credentials by clicking the Authorize button.
I get an error and can't log in

Output of `uname -a` or `ver`

No response

Output of `java -version`

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

3.3.0

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

No response

Additional information

No response

quarkus-bot[bot] commented 1 year ago

/cc @pedroigor (keycloak), @sberyozkin (keycloak)

sberyozkin commented 1 year ago

@tmulle Thanks, let me clarify:

Devmode, when you have Quarkus launching its own Keycloak container

SwaggerUI has its own script logic and can't be used independently to login to Keycloak DevService, you have to use Swagger UI from inside OpenId Connect card, you select the provider link, login to Keycloak, and there will be a Swagger UI link there:

https://quarkus.io/guides/security-openid-connect-dev-services#test-with-swagger-graphql

The token acquired as part of the SPA login (or if you configure Keycloak devservices, the client credentials grant, https://quarkus.io/guides/security-openid-connect-dev-services#client-credentials-grant) will be wired in via a local storage hack, to Swagger UI. Click on Swagger UI link and start testing, just avoid Swagger UI Authorize option as it will ignore and reset the token acquired by DevUI

Prod - any issues related to Swagger UI and prod instance of Keycloak are not related to Quarkus, indeed, please follow up with them in the Keycloak repository.

I think, as far as this issue is concerned, once you confirm you can use Swagger UI from the OIDC card as described above, we can close the issue, can you please confirm it ?

sberyozkin commented 1 year ago

@tmulle

SwaggerUI has its own script logic and can't be used independently to login to Keycloak DevService,

Well it might work, but only the DevService for Keycloak with the Swagger UI used from inside the OIDC card as described above guarantees it will work with the default dev services container.

Direct use of Swagger UI is a Swagger UI specific and/or Keycloak issue if a given grant authentication does not work

tmulle commented 1 year ago

@sberyozkin

Ok thanks.. I see the new UI you mentioned.

However, when I set it up for client_credentials and enter my valid client_id and client_secret and click the Test button, it always fails with an error "Unauthorized Client or Credentials"

I know they are good because I can log in using Postman with the same URL and credentials the DEV service UI is trying to use.

I can use Postman to get the access token and use it in my backend services just fine. So I know the creds are good.

Debugging the code below, the call is returning a 401 Unauthorized.

I tried different paths of '/' , '/*' and even an exact path I hit in Postman and nothing works. The instructions aren't clear what kind of service endpoint I need to use.

It fails with the error:

2023-08-28 18:16:27,048 ERROR [io.qua.dev.run.jso.JsonRpcCodec] (vert.x-eventloop-thread-2) Error in JsonRPC Call: java.lang.RuntimeException: {"error":"unauthorized_client","error_description":"Invalid client or Invalid client credentials"}
        at io.quarkus.oidc.runtime.devui.OidcDevServicesUtils.getAccessTokenFromJson(OidcDevServicesUtils.java:219)
        at io.quarkus.oidc.runtime.devui.OidcDevServicesUtils.lambda$getClientCredAccessToken$1(OidcDevServicesUtils.java:87)
        at io.smallrye.context.impl.wrappers.SlowContextualFunction.apply(SlowContextualFunction.java:21)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransform$UniOnItemTransformProcessor.onItem(UniOnItemTransform.java:36)
        at io.smallrye.mutiny.vertx.AsyncResultUni.lambda$subscribe$1(AsyncResultUni.java:35)
        at io.smallrye.mutiny.vertx.DelegatingHandler.handle(DelegatingHandler.java:25)
        at io.vertx.ext.web.client.impl.HttpContext.handleDispatchResponse(HttpContext.java:397)
        at io.vertx.ext.web.client.impl.HttpContext.execute(HttpContext.java:384)
        at io.vertx.ext.web.client.impl.HttpContext.next(HttpContext.java:362)
        at io.vertx.ext.web.client.impl.HttpContext.fire(HttpContext.java:329)
        at io.vertx.ext.web.client.impl.HttpContext.dispatchResponse(HttpContext.java:291)
        at io.vertx.ext.web.client.impl.HttpContext.lambda$null$7(HttpContext.java:507)
        at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
        at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:246)
        at io.vertx.core.impl.EventLoopContext.lambda$runOnContext$0(EventLoopContext.java:43)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)

Any ideas?

pedroigor commented 1 year ago

@tmulle That is how partial export should work to avoid leaking sensitive data at runtime. You can use the export command to run a full export and achieve what you want. When running a full export it is safer to assume the user intention. As you noticed, the behavior has been there (correctly, IMO) for a long time.

sberyozkin commented 1 year ago

@tmulle I've tried both client (client_credentials) and password grants in Dev UI in quarkus-quickstarts/security-openid-connect-quickstart and in both cases the token can be acquired, though for some reasons can't be verified, I'll have a look

sberyozkin commented 1 year ago

@tmulle I've fixed the typo related to the client credential or password token verification when Dev UI is used to acquire the tokens. However, you are saying that you can't even get the token from OIDC Dev UI with the client credentials grant.

However, when I set it up for client_credentials and enter my valid client_id and client_secret and click the Test button, it always fails with an error "Unauthorized Client or Credentials"

This client id and secret should already be setup by Dev UI - these are values which are configured in application.properties or Dev UI will generate them itself if none are setup. Using some other clientid/secret pairs which may be available in the Keycloak realm is not currently supported but I'll have a look

sberyozkin commented 1 year ago

@tmulle

Using some other clientid/secret pairs which may be available in the Keycloak realm is not currently supported but I'll have a look

It actually works as expected, if one enters custom client/secret they are passed to Keycloak correctly.

However, when I set it up for client_credentials and enter my valid client_id and client_secret and click the Test button, it always fails with an error "Unauthorized Client or Credentials"

Can you share your application.properties, and custom realm, and what clientd/secret you type ? (I'm assuming it is a test realm, if not, please create the one)

sberyozkin commented 1 year ago

@tmulle Is either the client id or secret which you enter has capital case letters ? If yes then #35687 will fix it

tmulle commented 1 year ago

So, I've tried everything and I still can't get it to work. I even tried resetting the client secret to lower case since my original one has mixture of upper and lower.

I tried adding the client-id and secret in my application properties and leaving them out.

The weird thing is that when I try to hit a service endpoint in my application that isn't authenticated I still get a 401.

Here is the debug log from the dev services when I try to log in using the OIDC provider link:

2023-09-01 13:12:19,937 INFO  [io.qua.oid.run.dev.OidcDevServicesUtils] (vert.x-eventloop-thread-3) Using a client_credentials grant to get a token token from 'http://localhost:32848/realms/license-server/protocol/openid-connect/token' with client id 'cli-app'
2023-09-01 13:12:19,986 INFO  [io.qua.oid.run.dev.OidcDevServicesUtils] (vert.x-eventloop-thread-3) Test token: eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICI3cWh1eml6WklyT25qNnVTR2R3ODlDLXdJMm1pak84VzR1eHpmM0stRVNBIn0.eyJleHAiOjE2OTM1ODg2MzksImlhdCI6MTY5MzU4ODMzOSwianRpIjoiNTZjMmY3ODQtMDk2OS00YzI2LTk0MjUtNGI5NDQxZDFhMmViIiwiaXNzIjoiaHR0cDovL3RtdWxsZS13cy5lbmcucmFqYW50LmNvbS9yZWFsbXMvbGljZW5zZS1zZXJ2ZXIiLCJhdWQiOiJhY2NvdW50Iiwic3ViIjoiNzMyZGY4YWItNjdmYS00NzhkLTkxMmQtZmFhNzc2M2UyZTNlIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoiY2xpLWFwcCIsImFjciI6IjEiLCJhbGxvd2VkLW9yaWdpbnMiOlsiLyoiXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbImRlZmF1bHQtcm9sZXMtbGljZW5zZS1zZXJ2ZXIiLCJsaWNlbnNlLWhpc3RvcnktcmVhZCIsImxpY2Vuc2UtYXNzaWduIiwibGljZW5zZS1jcmVhdGUiLCJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIiwibGljZW5zZS10cmFuc2ZlciJdfSwicmVzb3VyY2VfYWNjZXNzIjp7ImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJtYW5hZ2UtYWNjb3VudC1saW5rcyIsInZpZXctcHJvZmlsZSJdfX0sInNjb3BlIjoiZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwiY2xpZW50SG9zdCI6IjE3Mi4xNy4wLjEiLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJzZXJ2aWNlLWFjY291bnQtY2xpLWFwcCIsImNsaWVudEFkZHJlc3MiOiIxNzIuMTcuMC4xIiwiY2xpZW50X2lkIjoiY2xpLWFwcCJ9.Mhr4y75S2wc_IazMM5yc_k4hqJIv2tdW1LIVUsQ1aJS5SDeRfgj3MZCLsM63d9pOpxi2rcYDubCwaG-voBLuL0J6IRWPkUmgctnalDOzzygpWW5nVvhyjCpHlckEoEHea5TFs7EmUGzv9yxx7-EYvUOMvhdGEEPboLM_GpNc3EpYCG5IgxbCry77zax-YHqFHT6Szmzn7MmElKT4x99QUIFOEqokHFP0u3CkpxG8nqCdAj44BhE-O_ocYCktZZ637wDsasUN7W6b5XYQmC-VteEMQ4P3A6lNkMVR7JLKtlBemmdkOiesGh8lX7lOL1G3fU4NHYkfQYQpvB_l5TBDAQ

2023-09-01 13:12:19,987 INFO  [io.qua.oid.run.dev.OidcDevServicesUtils] (vert.x-eventloop-thread-3) Sending token to 'http://localhost:8080/api/v1/license/906169af-116b-4342-a982-df76c551fa11'
2023-09-01 13:12:19,998 INFO  [io.qua.oid.run.dev.OidcDevServicesUtils] (vert.x-eventloop-thread-3) Result: 401

I'll attach my realm I'm testing.. it's a service user with client-id cli-app.

license-server-realm.txt

This is the OIDC portion of my application.properties:

I don't have a client-id or client-secret defined because my quarkus application just receives Bearer tokens.


# The OpenID Connect (OIDC) flow, which Keycloak uses,
# relies on redirects, and those redirects need to match the URLs
# that are registered with the OIDC provider, in this case, Keycloak.
# If you registered "localhost" as the URL when you set up your client
# in Keycloak and then try to use an IP address in the Quarkus OIDC configuration,
# the URLs will not match, causing the token validation to fail.
#
# Keycloak (note this needs to be an ip address because
#The issuer in the token: Keycloak puts the URL of the server, the issuer, in the issued tokens.
# The Quarkus OIDC extension validates the issuer.
# If the issuer in the token does not match the OIDC server URL configured
# in your Quarkus application, validation will fail.
#quarkus.oidc.auth-server-url=http://172.20.21.1:8888/realms/rajant
%prod.quarkus.oidc.auth-server-url=http://${KEYCLOAK_SERVER_PORT:localhost:40002}/realms/${KEYCLOAK_REALM:license-server}
#quarkus.oidc.client-id=${OIDC_CLIENT:cli-app}
#quarkus.oidc.credentials.secret=${OIDC_SECRET:secret}
#quarkus.oidc.roles.role-claim-path=${OIDC_ROLES_CLAIM_PATH:resource_access/license-server-backend/roles}
quarkus.keycloak.devservices.realm-path=license-server-realm.json
quarkus.oidc.devui.grant.type=client```

sberyozkin commented 1 year ago

@tmulle OK, thanks, so the logs show that you can actually acquire the token, but you must be hitting the unfortunate typo which was fixed, #35685, Quarkus 3.3.2 will be released soon, so once it is, please retry and it should work. Lets keep the issue open until you confirm it works, cheers

tmulle commented 1 year ago

@sberyozkin ok it looks like 3.3.2 partially fixes my issue. There is still the password issue with Upper/Lowercase.

1) If I use the password "secret" for my client things work (partially). But if I try something like "PassWord!" it doesn't work. It tells me invalid client or credentials but I see them clearly in Keycloak UI.

2023-09-06 10:11:50,828 INFO  [io.qua.oid.run.dev.OidcDevServicesUtils] (vert.x-eventloop-thread-3) Using a client_credentials grant to get a token token from 'http://localhost:63533/realms/license-server/protocol/openid-connect/token' with client id 'cli-app'
2023-09-06 10:11:50,887 ERROR [io.qua.oid.run.dev.OidcDevServicesUtils] (vert.x-eventloop-thread-3) Token can not be acquired from OpenId Connect provider: java.lang.RuntimeException: {"error":"unauthorized_client","error_description":"Invalid client or Invalid client credentials"}
2023-09-06 10:11:50,893 ERROR [io.qua.dev.run.jso.JsonRpcCodec] (vert.x-eventloop-thread-3) Error in JsonRPC Call: java.lang.RuntimeException: {"error":"unauthorized_client","error_description":"Invalid client or Invalid client credentials"}
        at io.quarkus.oidc.runtime.devui.OidcDevServicesUtils.getAccessTokenFromJson(OidcDevServicesUtils.java:219)
        at io.quarkus.oidc.runtime.devui.OidcDevServicesUtils.lambda$getClientCredAccessToken$1(OidcDevServicesUtils.java:87)
        at io.smallrye.context.impl.wrappers.SlowContextualFunction.apply(SlowContextualFunction.java:21)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransform$UniOnItemTransformProcessor.onItem(UniOnItemTransform.java:36)
        at io.smallrye.mutiny.vertx.AsyncResultUni.lambda$subscribe$1(AsyncResultUni.java:35)
        at io.smallrye.mutiny.vertx.DelegatingHandler.handle(DelegatingHandler.java:25)
        at io.vertx.ext.web.client.impl.HttpContext.handleDispatchResponse(HttpContext.java:397)

2) The Swagger UI link in the OIDC Provider screen doesn't work. Once I get the token and click it it tells me Unknown Error. So, if I go into the Dev UI - Swagger UI and try an authorized endpoint it appears the token is working.

sberyozkin commented 1 year ago

Hi @tmulle #35888 fixes the SwaggerUI problem for client creds, this one in particular is not used often from DevUI, so a few typos introduced during the migration were not caught, so thanks for catching them.

The capital case password problem can not be reproduced. I've tried both code flow and client creds, you can confirm it with empty application properties and

quarkus.oidc.credentials.secret=PassWord!
quarkus.keycloak.devservices.grant.type=client

in DevUI, follow Keycloak Admin link, login as admin:admin, confirm a quarkus-app client's password is PassWord!, now go to SPA, and try the client credential form.

Note #35888 will resolve this issue once merged. If something still does not work after it is merged then please open more specific issues, it will be easier to handle them.

Thanks

quarkusio / quarkus