keycloak / keycloak-benchmark

Keycloak Benchmark
https://www.keycloak.org/keycloak-benchmark/
Apache License 2.0
127 stars 70 forks source link

Change to Argon2 password hashing #744

Closed ahus1 closed 3 months ago

ahus1 commented 5 months ago

Description

With https://github.com/keycloak/keycloak/issues/27990 in place, Argon2 will be the new hashing method in all non-FIPS environments.

Discussion

No response

Motivation

It is a lot faster, and would allow use to set up our environments faster. We could also again test with 100_000 users instead of only 20_000. For KC25, we also need the new benchmarks.

Details

At the moment the defaults of the hashing are hardcoded in the dataset module. We should remove all references to those hardcoded values from there and rely on Keycloak's defaults instead.

ahus1 commented 4 months ago

I'm reopening this issue as not all items seem to be complete in this issue:

kami619 commented 4 months ago

We seem to get this WARN message when we are using the default hashing iteration to create users, seems to be logged for each user created. We should tune this to be minimal, esp. while using a default hashing iteration.

{"timestamp":"2024-03-29T16:08:00.956988593Z","sequence":18600,"loggerClassName":"org.jboss.logging.Logger","loggerName":"org.keycloak.crypto.hash.Argon2PasswordHashProvider","level":"WARN","message":"Iterations for Argon should be less than 100, using default","threadName":"pool-9-thread-16","threadId":165,"mdc":{},"ndc":"","hostName":"keycloak-2","processName":"QuarkusEntryPoint","processId":1}
kami619 commented 4 months ago

I repeated a test from the past with the below config and it didn't yield good results.

Gatling result:

Screenshot 2024-03-29 at 14 22 58

Keycloak Config:

KC_INSTANCES=3
KC_CPU_REQUESTS=6
KC_INSTANCES=3
KC_DISABLE_STICKY_SESSION=true
KC_MEMORY_REQUESTS_MB=3000
KC_MEMORY_LIMITS_MB=4000
KC_DB_POOL_INITIAL_SIZE=30
KC_DB_POOL_MAX_SIZE=30
KC_DB_POOL_MIN_SIZE=30
KC_DATABAS=aurora-postgres

Load Test command:

./benchmark.sh eu-west-1 --scenario=keycloak.scenario.authentication.AuthorizationCode \
--server-url=https://client.gh-keycloak-a-gh-keycloak-b-XXXXX.keycloak-benchmark.com \
--realm-name=realm-0 --users-per-sec=200 --ramp-up=20 --logout-percentage=100 \
--measurement=600 --users-per-realm=100000 \
--log-http-on-failure --sla-error-percentage=0.001
kami619 commented 4 months ago

Against the same Keycloak config, ran another one but with just 40 users-per-sec, and that seems to have no issues.

Gatling result:

Screenshot 2024-03-29 at 14 26 50

Load Test command:

./benchmark.sh eu-west-1 --scenario=keycloak.scenario.authentication.AuthorizationCode \
--server-url=https://client.gh-keycloak-a-gh-keycloak-b-xxxxx.keycloak-benchmark.com \
--realm-name=realm-0 --users-per-sec=40 --ramp-up=20 --logout-percentage=100 \
--measurement=600 --users-per-realm=100000 \
--log-http-on-failure --sla-error-percentage=0.001
ahus1 commented 3 months ago

There is issue https://github.com/keycloak/keycloak/issues/29033 to change Keycloak's GC settings

ahus1 commented 3 months ago

Remove duplicate contents from https://www.keycloak.org/keycloak-benchmark/benchmark-guide/latest/report/rosa-benchmark-key-results and link to main Keycloak site