Run test-suite with Keycloak-X

thomasdarimont commented 2 years ago

Description

Add support for running the testsuite with the Quarkus based Keycloak.X distribution instead of Wildfly based Keycloak.

Discussion

No response

Motivation

The Keycloak project is going to declare the Quarkus based Keycloak distribution (keycloak.x) as the default distribution soon. Although there are apparently no API differences between the Wildfly based Keycloak distribution and Keycloak-X, it could be helpful to use the Keycloak.X distribution as the base for tests in the long run.

Details

It should be possible to easily switch between the Wildfly (WF) based Keycloak distribution and Keycloak.X. This could be achived by externalizing the keycloak service from the docker-compose.yml and creating two separate additional configuration files for WF based Keycloak and Quarkus based Keycloak.X.

Running the setup with Keycloak (WF)

docker-compose -f docker-compose.yml -f docker-compose-keycloak.yml up --build --remove-orphans

Running the setup with Keycloak.X (Quarkus)

docker-compose -f docker-compose.yml -f docker-compose-keycloakx.yml up --build --remove-orphans

I have an initial example for this where I can start Keycloak.x with https and the required SPI settings and certs imported into the cacerts truststore. The only thing missing is the realm import during startup. Keycloak.X doesn't seem to support importing json files at startup anymore. Here we could use an additional docker compose service that waits for Keycloak to become ready and then uses the kc.sh cli to import the realm config afterwards.

If you are interested in this let me know, then I'll create a PR with the implementation for the above :)

thomasdarimont commented 2 years ago

I have a "working" version with keycloak.x with realm import via side-car in my branch: https://github.com/thomasdarimont/kc-sig-fapi/tree/poc/use-keycloak-x

Working means:

Keycloak-X container starts
server certs and client-ca certs are imported in the truststore
SPI config customizations are in place
Keycloak is accessible via https
TLS version and cipher settings are in place
Test realm json is imported once Keycloak service is "healthy"

To start just Keycloak-X with realm import run:

docker-compose -f docker-compose.yml -f docker-compose-keycloakx.yml up --build --remove-orphans keycloak keycloak-import

To start the whole testsuite with Keycloak-X

docker-compose -f docker-compose.yml -f docker-compose-keycloakx.yml up --build --remove-orphans

Changes are shown here: https://github.com/keycloak/kc-sig-fapi/compare/main...thomasdarimont:poc/use-keycloak-x?expand=1

tnorimat commented 2 years ago

@thomasdarimont Thank you very much. I would like to check your work.

tnorimat commented 2 years ago

@thomasdarimont Hello, I've checkouted your branch and started just Keycloak-X with realm import run as you mentioned. After terminating and restarting it, the following error occurred.

ERROR: for 008365e3df3a_kc-sig-fapi_httpd_1  Cannot start service httpd: driver failed programming external connectivity on endpoint 008365e3df3a_kc-sig-fapi_httpd_1 (00b3d8ffe8737784dcd4792ffcdec4768136cfad259b27c8fabc9348afca0607): Bind for 0.0.0.0:8443 failed: port is already allocated

It might happens due to my environment. Could you know thie error?

thomasdarimont commented 2 years ago

Thanks for looking into this! You probably need to remove the - "8443:8443" port binding in docker-compose-keycloakx.yml for the keycloak container. I left that in while I was testing the Keycloak-X setup.

To verify that Keycloak-x can start you could try this:

docker-compose -f docker-compose.yml -f docker-compose-keycloakx.yml up --build --remove-orphans keycloak keycloak-import

tnorimat commented 2 years ago

@thomasdarimont To run conformance test properly, the following modification might be needed.

keycloak/Dockerfile.x:

ENV KC_HTTPS_PORT=9443

In this conformance test run environment, keycloak needs to open 9443 port, not 8443 port as default. 8443 is used by conformance test server.

docker-compose-keycloakx.yml:

      - "--hostname=${KEYCLOAK_FQDN}"

If adding port number here, conformance test does not run properly.

Considering that keycloak 17 has been released, we might use keycloak 17.

.env

KEYCLOAK_BASE_IMAGE=${KEYCLOAK_IMAGE:-quay.io/keycloak/keycloak:17.0.0-legacy}
KEYCLOAKX_BASE_IMAGE=${KEYCLOAKX_IMAGE:-quay.io/keycloak/keycloak:17.0.0}

Could you check the modification above?

tnorimat commented 2 years ago

By the way, some test (checking "acr" claim in ID Token) fails by kc16 and kc17. The change from kc15 to kc16 causes the problem.

tnorimat commented 2 years ago

@thomasdarimont Could you send PR of your PoC branch? I would like to continue working on running test-suite with Keycloak-X based on your PR.

thomasdarimont commented 2 years ago

@tnorimat thank you for your valuable feedback :)

I just created a new with your recommended changes: https://github.com/keycloak/kc-sig-fapi/pull/248

adapted the ports
updated the images to Keycloak 17.0.0

tnorimat commented 2 years ago

@thomasdarimont I've confirmed that FAPI 1.0 Advanced, FAPI-CIBA, Open Banking Brazil FAPI 1.0 conformance tests have passed against both quarkus based keycloak 17 and wildfly based keycloak 17 by my additional PR based on your PR. Therefore, I close this issue. Thank you very much.

keycloak / kc-sig-fapi