search

bitnami / charts

Bitnami Helm Charts
https://bitnami.com
Other
8.62k stars 8.99k forks source link

keycloak add cacert value or better documentation using existing values #7536

Closed karezza closed 2 years ago

karezza commented 2 years ago

Which chart: keycloak

Is your feature request related to a problem? Please describe. To use LDAPS in a Windows AD environment the onprem cacert must be installed.

Describe the solution you'd like A volume to mount with the cacert.pem file which will be added to the keystore automatically.

Describe alternatives you've considered Before using kubernetes I have performed this in the past using 'docker exec' by running:

keytool -importcert -file ./cacert.pem -keystore /usr/lib/jvm/java-11-openjdk-11.0.11.0.9-0.el8_3.x86_64/lib/security/cacerts

From ticket browsing I thought this might work:

$ cat ./values.yaml
auth.tls.enabled: true
auth.tls.existingSecret: keycloak-cacert

$ k describe secrets keycloak-cacert 
Name:         keycloak-cacert
Namespace:    keycloak
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
keycloak-0.keystore.jks:  1719 bytes
keycloak.truststore.jks:  1719 bytes

Additional context The attempt above failed with CrashLoopBackoff.

randradas commented 2 years ago

Hello @karezza thank you for the issue!, I would like to ask you for some things in order to help me to reproduce the problem:

Thank you!

karezza commented 2 years ago

$ cat ./upgrade-install.sh 

#!/bin/bash

NAME=keycloak
NS=$NAME
REPO=https://charts.bitnami.com/bitnami
REPO_ALIAS=bitnami
CHART=keycloak

# prep
helm repo add $REPO_ALIAS $REPO
helm repo update
touch values.yaml

# if an upgrade
export ADMIN_PASSWORD=$(kubectl get secret --namespace "keycloak" keycloak -o jsonpath="{.data.admin-password}" | base64 --decode)

# install/upgrade
helm upgrade \
        --install \
        --namespace $NS \
        --create-namespace \
        -f ./values.yaml \
        --set auth.adminPassword=$ADMIN_PASSWORD \
        $NAME $REPO_ALIAS/$CHART

$ cat ./values.yaml

image.debug: false
auth.tls.enabled: true
auth.tls.existingSecret: keycloak-cacert

$ k describe pod keycloak-0

Events:
  Type     Reason     Age                     From     Message
  ----     ------     ----                    ----     -------
  Warning  Unhealthy  17m (x19 over 149m)     kubelet  Readiness probe failed: Get "http://172.21.162.186:8080/auth/realms/master": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

$ k logs -f keycloak-0

keycloak 17:15:08.83 INFO  ==> ** Starting keycloak **
=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /opt/bitnami/keycloak

  JAVA: /opt/bitnami/java/bin/java

  JAVA_OPTS:  -server -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true  --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED --add-exports=jdk.unsupported/sun.reflect=ALL-UNNAMED

=========================================================================

17:15:09,655 INFO  [org.jboss.modules] (main) JBoss Modules version 1.11.0.Final
17:15:10,203 INFO  [org.jboss.msc] (main) JBoss MSC version 1.4.12.Final
17:15:10,225 INFO  [org.jboss.threads] (main) JBoss Threads version 2.4.0.Final
17:15:10,308 INFO  [org.jboss.as] (MSC service thread 1-2) WFLYSRV0049: Keycloak 15.0.2 (WildFly Core 15.0.1.Final) starting
17:15:11,007 INFO  [org.wildfly.security] (ServerService Thread Pool -- 21) ELY00001: WildFly Elytron version 1.15.3.Final
17:15:11,333 INFO  [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 18) WFLYCTL0033: Extension 'security' is deprecated and may not be supported in future versions
17:15:11,574 INFO  [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/core-service=management/management-interface=http-interface' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
17:15:11,600 INFO  [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 20) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/subsystem=undertow/server=default-server/https-listener=https' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
17:15:11,669 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 2) WFLYDR0001: Content added at location /opt/bitnami/keycloak/standalone/data/content/52/f767f1c27867fe6ab51811d9ec201c0802f93c/content
17:15:11,774 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
17:15:11,790 INFO  [org.xnio] (MSC service thread 1-2) XNIO version 3.8.4.Final
17:15:11,796 INFO  [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.8.4.Final
17:15:11,830 INFO  [org.wildfly.extension.health] (ServerService Thread Pool -- 38) WFLYHEALTH0001: Activating Base Health Subsystem
17:15:11,831 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 43) WFLYCLJG0001: Activating JGroups subsystem. JGroups version 4.2.11
17:15:11,848 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 39) WFLYCLINF0001: Activating Infinispan subsystem.
17:15:11,852 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 53) WFLYSEC0002: Activating Security Subsystem
17:15:11,852 INFO  [org.wildfly.extension.metrics] (ServerService Thread Pool -- 48) WFLYMETRICS0001: Activating Base Metrics Subsystem
17:15:11,861 INFO  [org.wildfly.extension.io] (ServerService Thread Pool -- 40) WFLYIO001: Worker 'default' has auto-configured to 2 IO threads with 16 max task threads based on your 1 available processors
17:15:11,880 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 50) WFLYNAM0001: Activating Naming Subsystem
17:15:11,886 INFO  [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 5.0.20.Final
17:15:11,898 INFO  [org.jboss.as.jaxrs] (ServerService Thread Pool -- 41) WFLYRS0016: RESTEasy version 3.15.1.Final
17:15:11,902 INFO  [org.jboss.as.security] (MSC service thread 1-2) WFLYSEC0001: Current PicketBox version=5.0.3.Final-redhat-00007
17:15:11,930 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 33) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.4)
17:15:11,962 INFO  [org.jboss.as.connector] (MSC service thread 1-1) WFLYJCA0009: Starting Jakarta Connectors Subsystem (WildFly/IronJacamar 1.4.27.Final)
17:15:11,967 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 33) WFLYJCA0005: Deploying non-JDBC-compliant driver class org.postgresql.Driver (version 42.2)
17:15:12,000 WARN  [org.wildfly.clustering.web.undertow] (ServerService Thread Pool -- 56) WFLYCLWEBUT0007: No routing provider found for default-server; using legacy provider based on static configuration
17:15:12,059 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 56) WFLYUT0014: Creating file handler for path '/opt/bitnami/keycloak/welcome-content' with options [directory-listing: 'false', follow-symlink: 'false', case-sensitive: 'true', safe-symlink-paths: '[]']
17:15:12,070 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0003: Undertow 2.2.5.Final starting
17:15:12,105 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0018: Started Driver service with driver-name = h2
17:15:12,106 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0018: Started Driver service with driver-name = postgresql
17:15:12,135 INFO  [org.jboss.as.ejb3] (MSC service thread 1-1) WFLYEJB0482: Strict pool mdb-strict-max-pool is using a max instance size of 4 (per class), which is derived from the number of CPUs on this host.
17:15:12,146 INFO  [org.jboss.as.ejb3] (MSC service thread 1-1) WFLYEJB0481: Strict pool slsb-strict-max-pool is using a max instance size of 16 (per class), which is derived from thread worker pool sizing.
17:15:12,150 INFO  [org.jboss.as.naming] (MSC service thread 1-1) WFLYNAM0003: Starting Naming Service
17:15:12,187 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-2) WFLYMAIL0001: Bound mail session [java:jboss/mail/Default]
17:15:12,213 WARN  [org.wildfly.extension.elytron] (MSC service thread 1-2) WFLYELY00023: KeyStore file '/opt/bitnami/keycloak/standalone/configuration/application.keystore' does not exist. Used blank.
17:15:12,387 WARN  [org.wildfly.extension.elytron] (MSC service thread 1-2) WFLYELY01084: KeyStore /opt/bitnami/keycloak/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self-signed certificate for host localhost
17:15:12,391 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0012: Started server default-server.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.wildfly.extension.elytron.SSLDefinitions (jar:file:/opt/bitnami/keycloak/modules/system/layers/base/org/wildfly/extension/elytron/main/wildfly-elytron-integration-15.0.1.Final.jar!/) to method com.sun.net.ssl.internal.ssl.Provider.isFIPS()
WARNING: Please consider reporting this to the maintainers of org.wildfly.extension.elytron.SSLDefinitions
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
17:15:12,431 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0006: Undertow AJP listener ajp listening on 0.0.0.0:8009
17:15:12,447 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0006: Undertow HTTP listener default listening on 0.0.0.0:8080
17:15:12,447 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) Queuing requests.
17:15:12,448 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0018: Host default-host starting
17:15:12,471 INFO  [org.jboss.as.patching] (MSC service thread 1-1) WFLYPAT0050: Keycloak cumulative patch ID is: base, one-off patches include: none
17:15:12,476 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 58) MODCLUSTER000001: Initializing mod_cluster version 1.4.3.Final
17:15:12,481 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 58) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364
17:15:12,493 WARN  [org.jboss.as.domain.management.security] (MSC service thread 1-2) WFLYDM0111: Keystore /opt/bitnami/keycloak/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost
17:15:12,540 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0027: Starting deployment of "keycloak-metrics-spi-2.5.1.jar" (runtime-name: "keycloak-metrics-spi-2.5.1.jar")
17:15:12,540 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-2) WFLYSRV0027: Starting deployment of "keycloak-server.war" (runtime-name: "keycloak-server.war")
17:15:12,544 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-1) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/bitnami/keycloak/standalone/deployments
17:15:12,544 INFO  [org.jboss.as.ejb3] (MSC service thread 1-1) WFLYEJB0493: Jakarta Enterprise Beans subsystem suspension complete
17:15:12,657 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0006: Undertow HTTPS listener https listening on 0.0.0.0:8443
17:15:12,775 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
17:15:12,775 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0001: Bound data source [java:jboss/datasources/KeycloakDS]
17:15:12,944 INFO  [org.keycloak.subsystem.server.extension.KeycloakProviderDeploymentProcessor] (MSC service thread 1-2) Deploying Keycloak provider: keycloak-metrics-spi-2.5.1.jar
17:15:13,063 WARN  [org.jgroups.protocols.UDP] (ServerService Thread Pool -- 58) JGRP000015: the send buffer of socket ManagedMulticastSocketBinding was set to 1.00MB, but the OS only allocated 212.99KB
17:15:13,063 WARN  [org.jgroups.protocols.UDP] (ServerService Thread Pool -- 58) JGRP000015: the receive buffer of socket ManagedMulticastSocketBinding was set to 20.00MB, but the OS only allocated 212.99KB
17:15:13,064 WARN  [org.jgroups.protocols.UDP] (ServerService Thread Pool -- 58) JGRP000015: the send buffer of socket ManagedMulticastSocketBinding was set to 1.00MB, but the OS only allocated 212.99KB
17:15:13,064 WARN  [org.jgroups.protocols.UDP] (ServerService Thread Pool -- 58) JGRP000015: the receive buffer of socket ManagedMulticastSocketBinding was set to 25.00MB, but the OS only allocated 212.99KB
17:15:16,074 INFO  [org.jgroups.protocols.pbcast.GMS] (ServerService Thread Pool -- 58) keycloak-0: no members discovered after 3003 ms: creating cluster as coordinator
17:15:16,484 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 58) ISPN000128: Infinispan version: Infinispan 'Corona Extra' 11.0.9.Final
17:15:16,526 INFO  [org.infinispan.PERSISTENCE] (ServerService Thread Pool -- 60) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.marshalling.jboss.JBossMarshaller'
17:15:16,554 INFO  [org.infinispan.PERSISTENCE] (ServerService Thread Pool -- 58) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
17:15:16,554 INFO  [org.infinispan.PERSISTENCE] (ServerService Thread Pool -- 59) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
17:15:16,554 INFO  [org.infinispan.PERSISTENCE] (ServerService Thread Pool -- 61) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
17:15:16,554 INFO  [org.infinispan.PERSISTENCE] (ServerService Thread Pool -- 62) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.marshalling.jboss.JBossMarshaller'
17:15:16,689 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 58) ISPN000078: Starting JGroups channel ejb
17:15:16,689 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 61) ISPN000078: Starting JGroups channel ejb
17:15:16,689 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 60) ISPN000078: Starting JGroups channel ejb
17:15:16,689 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 59) ISPN000078: Starting JGroups channel ejb
17:15:16,689 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 62) ISPN000078: Starting JGroups channel ejb
17:15:16,695 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 60) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
17:15:16,695 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 58) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
17:15:16,695 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 59) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
17:15:16,695 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 61) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
17:15:16,696 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 62) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
17:15:16,705 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 60) ISPN000079: Channel ejb local address is keycloak-0, physical addresses are [172.21.162.144:55200]
17:15:16,705 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 59) ISPN000079: Channel ejb local address is keycloak-0, physical addresses are [172.21.162.144:55200]
17:15:16,705 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 58) ISPN000079: Channel ejb local address is keycloak-0, physical addresses are [172.21.162.144:55200]
17:15:16,705 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 62) ISPN000079: Channel ejb local address is keycloak-0, physical addresses are [172.21.162.144:55200]
17:15:16,706 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 61) ISPN000079: Channel ejb local address is keycloak-0, physical addresses are [172.21.162.144:55200]
17:15:16,745 INFO  [org.infinispan.CONFIG] (MSC service thread 1-1) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
17:15:16,746 INFO  [org.infinispan.CONFIG] (MSC service thread 1-1) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
17:15:16,962 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 59) WFLYCLINF0002: Started http-remoting-connector cache from ejb container
17:15:17,006 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 60) WFLYCLINF0002: Started offlineSessions cache from keycloak container
17:15:17,009 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started sessions cache from keycloak container
17:15:17,015 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 69) WFLYCLINF0002: Started offlineClientSessions cache from keycloak container
17:15:17,016 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 62) WFLYCLINF0002: Started authenticationSessions cache from keycloak container
17:15:17,011 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 58) WFLYCLINF0002: Started work cache from keycloak container
17:15:17,010 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0002: Started actionTokens cache from keycloak container
17:15:17,019 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 68) WFLYCLINF0002: Started loginFailures cache from keycloak container
17:15:17,022 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 70) WFLYCLINF0002: Started clientSessions cache from keycloak container
17:15:17,047 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 61) WFLYCLINF0002: Started users cache from keycloak container
17:15:17,047 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started authorization cache from keycloak container
17:15:17,047 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0002: Started keys cache from keycloak container
17:15:17,047 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started realms cache from keycloak container
17:15:17,326 WARN  [org.jboss.as.server.deployment] (MSC service thread 1-2) WFLYSRV0273: Excluded subsystem webservices via jboss-deployment-structure.xml does not exist.
17:15:17,891 INFO  [org.keycloak.services] (ServerService Thread Pool -- 63) KC-SERVICES0001: Loading config from standalone.xml or domain.xml
17:15:18,306 INFO  [org.keycloak.url.DefaultHostnameProviderFactory] (ServerService Thread Pool -- 63) Frontend: <request>, Admin: <frontend>, Backend: <request>
17:15:18,442 WARN  [org.keycloak.services] (ServerService Thread Pool -- 63) KC-SERVICES0047: metrics (org.jboss.aerogear.keycloak.metrics.MetricsEndpointFactory) is implementing the internal SPI realm-restapi-extension. This SPI is internal and may change without notice
17:15:18,446 WARN  [org.keycloak.services] (ServerService Thread Pool -- 63) KC-SERVICES0047: metrics-listener (org.jboss.aerogear.keycloak.metrics.MetricsEventListenerFactory) is implementing the internal SPI eventsListener. This SPI is internal and may change without notice
17:15:18,478 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started realmRevisions cache from keycloak container
17:15:18,482 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started userRevisions cache from keycloak container
17:15:18,486 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started authorizationRevisions cache from keycloak container
17:15:18,487 INFO  [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (ServerService Thread Pool -- 63) Node name: keycloak-0, Site name: null
17:15:19,390 WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (ServerService Thread Pool -- 63) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:345)
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:352)
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:287)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.createConnectionEventListener(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:1322)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.getConnection(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:499)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:632)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:604)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:624)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:440)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:789)
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151)
        at org.jboss.as.connector@23.0.2.Final//org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.getConnection(DefaultJpaConnectionProviderFactory.java:385)
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.lazyInit(LiquibaseDBLockProvider.java:65)
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.lambda$waitForLock$2(LiquibaseDBLockProvider.java:96)
        at org.keycloak.keycloak-server-spi-private@15.0.2//org.keycloak.models.utils.KeycloakModelUtils.suspendJtaTransaction(KeycloakModelUtils.java:758)
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.waitForLock(LiquibaseDBLockProvider.java:94)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.resources.KeycloakApplication$1.run(KeycloakApplication.java:134)
        at org.keycloak.keycloak-server-spi-private@15.0.2//org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:250)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.resources.KeycloakApplication.startup(KeycloakApplication.java:128)
        at org.keycloak.keycloak-wildfly-extensions@15.0.2//org.keycloak.provider.wildfly.WildflyPlatform.onStartup(WildflyPlatform.java:36)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:114)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:152)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.spi.ResteasyProviderFactory.createProviderInstance(ResteasyProviderFactory.java:2835)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:376)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.spi.ResteasyDeployment.startInternal(ResteasyDeployment.java:288)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:98)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:140)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:42)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:117)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.security.RunAsLifecycleInterceptor.init(RunAsLifecycleInterceptor.java:78)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:103)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.start(ManagedServlet.java:305)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ManagedServlet.createServlet(ManagedServlet.java:145)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:588)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:559)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:601)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:97)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:78)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.JBossThread.run(JBossThread.java:513)
Caused by: org.postgresql.util.PSQLException: FATAL: password authentication failed for user "bn_keycloak"
        at org.postgresql.jdbc@42.2.14//org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:525)
        at org.postgresql.jdbc@42.2.14//org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:146)
        at org.postgresql.jdbc@42.2.14//org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:197)
        at org.postgresql.jdbc@42.2.14//org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
        at org.postgresql.jdbc@42.2.14//org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:217)
        at org.postgresql.jdbc@42.2.14//org.postgresql.Driver.makeConnection(Driver.java:458)
        at org.postgresql.jdbc@42.2.14//org.postgresql.Driver.connect(Driver.java:260)
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:321)
        ... 57 more

17:15:19,397 FATAL [org.keycloak.services] (ServerService Thread Pool -- 63) Error during startup: java.lang.RuntimeException: Failed to connect to database
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.getConnection(DefaultJpaConnectionProviderFactory.java:391)
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.lazyInit(LiquibaseDBLockProvider.java:65)
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.lambda$waitForLock$2(LiquibaseDBLockProvider.java:96)
        at org.keycloak.keycloak-server-spi-private@15.0.2//org.keycloak.models.utils.KeycloakModelUtils.suspendJtaTransaction(KeycloakModelUtils.java:758)
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.waitForLock(LiquibaseDBLockProvider.java:94)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.resources.KeycloakApplication$1.run(KeycloakApplication.java:134)
        at org.keycloak.keycloak-server-spi-private@15.0.2//org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:250)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.resources.KeycloakApplication.startup(KeycloakApplication.java:128)
        at org.keycloak.keycloak-wildfly-extensions@15.0.2//org.keycloak.provider.wildfly.WildflyPlatform.onStartup(WildflyPlatform.java:36)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:114)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:152)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.spi.ResteasyProviderFactory.createProviderInstance(ResteasyProviderFactory.java:2835)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:376)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.spi.ResteasyDeployment.startInternal(ResteasyDeployment.java:288)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:98)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:140)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:42)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:117)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.security.RunAsLifecycleInterceptor.init(RunAsLifecycleInterceptor.java:78)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:103)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.start(ManagedServlet.java:305)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ManagedServlet.createServlet(ManagedServlet.java:145)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:588)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:559)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:601)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:97)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:78)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.JBossThread.run(JBossThread.java:513)
Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/KeycloakDS
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:159)
        at org.jboss.as.connector@23.0.2.Final//org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
        at org.keycloak.keycloak-model-jpa@15.0.2//org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.getConnection(DefaultJpaConnectionProviderFactory.java:385)
        ... 45 more
Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/KeycloakDS
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:690)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:440)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:789)
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151)
        ... 47 more
Caused by: javax.resource.ResourceException: IJ031084: Unable to create connection
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:345)
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:352)
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:287)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.createConnectionEventListener(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:1322)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.getConnection(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:499)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:632)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:604)
        at org.jboss.ironjacamar.impl@1.4.27.Final//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:624)
        ... 50 more
Caused by: org.postgresql.util.PSQLException: FATAL: password authentication failed for user "bn_keycloak"
        at org.postgresql.jdbc@42.2.14//org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:525)
        at org.postgresql.jdbc@42.2.14//org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:146)
        at org.postgresql.jdbc@42.2.14//org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:197)
        at org.postgresql.jdbc@42.2.14//org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
        at org.postgresql.jdbc@42.2.14//org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:217)
        at org.postgresql.jdbc@42.2.14//org.postgresql.Driver.makeConnection(Driver.java:458)
        at org.postgresql.jdbc@42.2.14//org.postgresql.Driver.connect(Driver.java:260)
        at org.jboss.ironjacamar.jdbcadapters@1.4.27.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:321)
        ... 57 more

17:15:19,409 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.keycloak.services.resources.KeycloakApplication
17:15:19,410 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002205: Adding provider class org.keycloak.services.error.KeycloakErrorHandler from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,411 INFO  [org.jboss.as.server] (Thread-1) WFLYSRV0220: Server shutdown has been requested via an OS signal
17:15:19,412 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002200: Adding class resource org.keycloak.services.resources.JsResource from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,412 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002200: Adding class resource org.keycloak.services.resources.ThemeResource from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,413 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002205: Adding provider class org.keycloak.services.filters.KeycloakSecurityHeadersFilter from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,413 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002220: Adding singleton resource org.keycloak.services.resources.RealmsResource from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,414 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002220: Adding singleton resource org.keycloak.services.resources.WelcomeResource from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,414 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002210: Adding provider singleton org.keycloak.services.util.ObjectMapperResolver from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,414 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002220: Adding singleton resource org.keycloak.services.resources.admin.AdminRoot from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,414 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 63) RESTEASY002220: Adding singleton resource org.keycloak.services.resources.RobotsResource from Application class org.keycloak.services.resources.KeycloakApplication
17:15:19,441 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/KeycloakDS]
17:15:19,444 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-1) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
17:15:19,444 INFO  [org.infinispan.manager.DefaultCacheManager] (ServerService Thread Pool -- 66) Stopping cache manager null on keycloak-0
17:15:19,444 INFO  [org.infinispan.manager.DefaultCacheManager] (ServerService Thread Pool -- 59) Stopping cache manager null on keycloak-0
17:15:19,444 INFO  [org.infinispan.manager.DefaultCacheManager] (ServerService Thread Pool -- 64) Stopping cache manager null on keycloak-0
17:15:19,449 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 64) ISPN000080: Disconnecting JGroups channel ejb
17:15:19,450 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 59) ISPN000080: Disconnecting JGroups channel ejb
17:15:19,449 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 66) ISPN000080: Disconnecting JGroups channel ejb
17:15:19,451 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0008: Undertow HTTPS listener https suspending
17:15:19,452 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0007: Undertow HTTPS listener https stopped, was bound to 0.0.0.0:8443
17:15:19,456 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0019: Stopped Driver service with driver-name = postgresql
17:15:19,462 INFO  [org.keycloak.subsystem.server.extension.KeycloakProviderDeploymentProcessor] (MSC service thread 1-1) Undeploying Keycloak provider: keycloak-metrics-spi-2.5.1.jar
17:15:19,466 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0028: Stopped deployment keycloak-metrics-spi-2.5.1.jar (runtime-name: keycloak-metrics-spi-2.5.1.jar) in 51ms
17:15:19,498 INFO  [org.hibernate.validator.internal.util.Version] (ServerService Thread Pool -- 63) HV000001: Hibernate Validator 6.0.22.Final
17:15:19,513 INFO  [org.jboss.resteasy.plugins.validation.i18n] (ServerService Thread Pool -- 63) RESTEASY008550: Unable to find CDI supporting ValidatorFactory. Using default ValidatorFactory
17:15:19,627 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 63) WFLYUT0021: Registered web context: '/auth' for server 'default-server'
17:15:19,629 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 63) WFLYUT0022: Unregistered web context: '/auth' from server 'default-server'
17:15:19,631 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 64) MODCLUSTER000002: Initiating mod_cluster shutdown
17:15:19,631 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0008: Undertow AJP listener ajp suspending
17:15:19,631 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0007: Undertow AJP listener ajp stopped, was bound to 0.0.0.0:8009
17:15:19,634 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
17:15:19,635 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0019: Stopped Driver service with driver-name = h2
17:15:19,638 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0019: Host default-host stopping
17:15:19,639 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0008: Undertow HTTP listener default suspending
17:15:19,640 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 0.0.0.0:8080
17:15:19,642 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0004: Undertow 2.2.5.Final stopping
17:15:19,646 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0028: Stopped deployment keycloak-server.war (runtime-name: keycloak-server.war) in 232ms
17:15:19,649 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 69) WFLYCLINF0003: Stopped realms cache from keycloak container
17:15:19,650 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 70) WFLYCLINF0003: Stopped keys cache from keycloak container
17:15:19,650 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0003: Stopped users cache from keycloak container
17:15:19,651 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0003: Stopped authorization cache from keycloak container
17:15:19,660 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 58) WFLYCLINF0003: Stopped loginFailures cache from keycloak container
17:15:19,660 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 59) WFLYCLINF0003: Stopped actionTokens cache from keycloak container
17:15:19,660 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0003: Stopped offlineClientSessions cache from keycloak container
17:15:19,660 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 61) WFLYCLINF0003: Stopped work cache from keycloak container
17:15:19,661 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0003: Stopped authenticationSessions cache from keycloak container
17:15:19,663 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 62) WFLYCLINF0003: Stopped offlineSessions cache from keycloak container
17:15:19,662 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 68) WFLYCLINF0003: Stopped clientSessions cache from keycloak container
17:15:19,664 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0003: Stopped sessions cache from keycloak container
17:15:19,664 INFO  [org.infinispan.manager.DefaultCacheManager] (ServerService Thread Pool -- 65) Stopping cache manager null on keycloak-0
17:15:19,665 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 60) WFLYCLINF0003: Stopped http-remoting-connector cache from ejb container
17:15:19,666 INFO  [org.infinispan.manager.DefaultCacheManager] (ServerService Thread Pool -- 60) Stopping cache manager null on keycloak-0
17:15:19,666 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 60) ISPN000080: Disconnecting JGroups channel ejb
17:15:19,671 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 65) ISPN000080: Disconnecting JGroups channel ejb
17:15:19,681 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([("subsystem" => "metrics")]): java.lang.NullPointerException
        at org.wildfly.extension.metrics@23.0.2.Final//org.wildfly.extension.metrics.MetricsSubsystemAdd$2.execute(MetricsSubsystemAdd.java:91)
        at org.jboss.as.controller@15.0.1.Final//org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:1040)
        at org.jboss.as.controller@15.0.1.Final//org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:779)
        at org.jboss.as.controller@15.0.1.Final//org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:468)
        at org.jboss.as.controller@15.0.1.Final//org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1415)
        at org.jboss.as.controller@15.0.1.Final//org.jboss.as.controller.ModelControllerImpl.boot(ModelControllerImpl.java:529)
        at org.jboss.as.controller@15.0.1.Final//org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:515)
        at org.jboss.as.controller@15.0.1.Final//org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:477)
        at org.jboss.as.server@15.0.1.Final//org.jboss.as.server.ServerService.boot(ServerService.java:459)
        at org.jboss.as.server@15.0.1.Final//org.jboss.as.server.ServerService.boot(ServerService.java:412)
        at org.jboss.as.controller@15.0.1.Final//org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:416)
        at java.base/java.lang.Thread.run(Thread.java:829)

17:15:19,683 ERROR [org.jboss.as.server] (ServerService Thread Pool -- 34) WFLYSRV0022: Deploy of deployment "keycloak-metrics-spi-2.5.1.jar" was rolled back with no failure message
17:15:19,693 ERROR [org.jboss.as.server] (ServerService Thread Pool -- 46) WFLYSRV0022: Deploy of deployment "keycloak-server.war" was rolled back with no failure message
17:15:19,710 INFO  [org.jboss.as] (MSC service thread 1-2) WFLYSRV0050: Keycloak 15.0.2 (WildFly Core 15.0.1.Final) stopped in 295ms
karezza commented 2 years ago

Looks like I had a corrupted pvc share, probably due to nfs file locking issues. Recreated and can now reach website. Specifying these values didn't cause it to crashloop:

auth.tls.enabled: true
auth.tls.existingSecret: keycloak-cacert

Not closing yet though, cause ldap still isn't working consistently yet. I'm not sure if the cacert is working. When I click test connection it sometimes works and sometimes doesn't. Testing ...

karezza commented 2 years ago

Still getting errors related to PKIX which usually are associated with a cacert not installed and unable to validate the certificate from the ldaps connection. I imported a realm configuration known to work on a standalone vm docker container install which I used exec to run the keytool command on, so the configuration should be correct.

I'm seeing these errors on the User Federation settings page for my realm. "Test connection" works consistently, but "Test authentication" fails consistently, along with the sync users buttons below. (see note above that I've duplicated a known to work configuration)

17:43:32,544 INFO  [org.keycloak.storage.ldap.LDAPIdentityStoreRegistry] (default task-14) Creating new LDAP Store for the LDAP storage provider: 'ldap', LDAP Configuration: {fullSyncPeriod=[604800], pagination=[true], connectionPooling=[true], usersDn=[CN=Users,DC=mydomain,DC=net], cachePolicy=[DEFAULT], useKerberosForPasswordAuthentication=[false], importEnabled=[true], enabled=[true], usernameLDAPAttribute=[sAMAccountName], bindDn=[CN=Travis Loyd,CN=Users,DC=mydomain,DC=net], changedSyncPeriod=[86400], lastSync=[1632332612], vendor=[ad], uuidLDAPAttribute=[objectGUID], allowKerberosAuthentication=[false], connectionUrl=[ldaps://sol.mydomain.net], syncRegistrations=[false], authType=[simple], debug=[false], searchScope=[2], useTruststoreSpi=[ldapsOnly], priority=[0], trustEmail=[false], userObjectClasses=[person, organizationalPerson, user], rdnLDAPAttribute=[cn], validatePasswordPolicy=[false], batchSizeForSync=[1000]}, binaryAttributes: []
17:43:32,546 INFO  [org.keycloak.storage.ldap.LDAPStorageProviderFactory] (default task-14) Sync changed users from LDAP to local store: realm: mydomain, federation provider: ldap, last sync time: Wed Sep 22 17:39:21 UTC 2021
17:43:33,039 ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default task-14) Uncaught server error: org.keycloak.models.ModelException: LDAP Query failed
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.idm.query.internal.LDAPQuery.getResultList(LDAPQuery.java:176)
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.LDAPStorageProviderFactory.syncImpl(LDAPStorageProviderFactory.java:530)
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.LDAPStorageProviderFactory.syncSince(LDAPStorageProviderFactory.java:489)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.managers.UserStorageSyncManager$3$1.call(UserStorageSyncManager.java:150)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.managers.UserStorageSyncManager$3$1.call(UserStorageSyncManager.java:143)
        at org.keycloak.keycloak-model-infinispan@15.0.2//org.keycloak.cluster.infinispan.InfinispanClusterProvider.executeIfNotExecuted(InfinispanClusterProvider.java:78)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.managers.UserStorageSyncManager$3.run(UserStorageSyncManager.java:143)
        at org.keycloak.keycloak-server-spi-private@15.0.2//org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:250)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.managers.UserStorageSyncManager.syncChangedUsers(UserStorageSyncManager.java:133)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.resources.admin.UserStorageProviderResource.syncUsers(UserStorageProviderResource.java:144)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:138)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:546)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:435)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:396)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:358)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:398)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:365)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:150)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:110)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:141)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:110)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:141)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:104)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:440)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:229)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:135)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:358)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:138)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:215)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:245)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:61)
        at org.jboss.resteasy.resteasy-jaxrs@3.15.1.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
        at javax.servlet.api@2.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
        at org.keycloak.keycloak-wildfly-extensions@15.0.2//org.keycloak.provider.wildfly.WildFlyRequestFilter.lambda$doFilter$0(WildFlyRequestFilter.java:41)
        at org.keycloak.keycloak-services@15.0.2//org.keycloak.services.filters.AbstractRequestFilter.filter(AbstractRequestFilter.java:43)
        at org.keycloak.keycloak-wildfly-extensions@15.0.2//org.keycloak.provider.wildfly.WildFlyRequestFilter.doFilter(WildFlyRequestFilter.java:39)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
        at io.undertow.core@2.2.5.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
        at io.undertow.core@2.2.5.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.core@2.2.5.Final//io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
        at io.undertow.core@2.2.5.Final//io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
        at io.undertow.core@2.2.5.Final//io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
        at io.undertow.core@2.2.5.Final//io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
        at io.undertow.core@2.2.5.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
        at io.undertow.core@2.2.5.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52)
        at io.undertow.core@2.2.5.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at org.wildfly.extension.undertow@23.0.2.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78)
        at io.undertow.servlet@2.2.5.Final//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99)
        at io.undertow.core@2.2.5.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
        at io.undertow.core@2.2.5.Final//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:841)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
        at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
        at org.jboss.xnio@3.8.4.Final//org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.keycloak.models.ModelException: Querying of LDAP failed org.keycloak.storage.ldap.idm.query.internal.LDAPQuery@5fadf703
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.idm.store.ldap.LDAPIdentityStore.fetchQueryResults(LDAPIdentityStore.java:289)
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.idm.query.internal.LDAPQuery.getResultList(LDAPQuery.java:172)
        ... 87 more
Caused by: javax.naming.CommunicationException: simple bind failed: sol.mydomain.net:636 [Root exception is javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target]
        at java.naming/com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:219)
        at java.naming/com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2895)
        at java.naming/com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:348)
        at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxFromUrl(LdapCtxFactory.java:262)
        at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:226)
        at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:280)
        at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:185)
        at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:115)
        at org.jboss.as.naming@23.0.2.Final//org.jboss.as.naming.InitialContext.getDefaultInitCtx(InitialContext.java:116)
        at org.jboss.as.naming@23.0.2.Final//org.jboss.as.naming.InitialContext.init(InitialContext.java:101)
        at java.naming/javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:154)
        at org.jboss.as.naming@23.0.2.Final//org.jboss.as.naming.InitialContext.<init>(InitialContext.java:91)
        at org.jboss.as.naming@23.0.2.Final//org.jboss.as.naming.InitialContextFactory.getInitialContext(InitialContextFactory.java:43)
        at java.naming/javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:730)
        at java.naming/javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:305)
        at java.naming/javax.naming.InitialContext.init(InitialContext.java:236)
        at java.naming/javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:154)
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.idm.store.ldap.LDAPContextManager.createLdapContext(LDAPContextManager.java:80)
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.idm.store.ldap.LDAPContextManager.getLdapContext(LDAPContextManager.java:100)
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.idm.query.internal.LDAPQuery.initPagination(LDAPQuery.java:211)
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.idm.store.ldap.LDAPOperationManager.searchPaginated(LDAPOperationManager.java:293)
        at org.keycloak.keycloak-ldap-federation@15.0.2//org.keycloak.storage.ldap.idm.store.ldap.LDAPIdentityStore.fetchQueryResults(LDAPIdentityStore.java:277)
        ... 88 more
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:349)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:287)
        at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
        at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
        at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
        at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
        at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
        at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
        at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1426)
        at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1336)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
        at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:841)
        at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1211)
        at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
        at java.base/java.io.BufferedOutputStream.flush(BufferedOutputStream.java:142)
        at java.naming/com.sun.jndi.ldap.Connection.writeRequest(Connection.java:414)
        at java.naming/com.sun.jndi.ldap.Connection.writeRequest(Connection.java:387)
        at java.naming/com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:359)
        at java.naming/com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:214)
        ... 109 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
        at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
        at java.base/sun.security.validator.Validator.validate(Validator.java:264)
        at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
        at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
        ... 127 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
        at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
        at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
        at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
        ... 133 more
randradas commented 2 years ago

Hello @karezza with all this info I will be able to reproduce it, I'll be back thank you!

karezza commented 2 years ago

Additional note, I can change ldaps to ldap and everything works, which verifies that I'm using a valid account. Also, that would seem to indicate that the certificates are correct, as they are being used when I click the "Test connection" button (I would think).

randradas commented 2 years ago

Still working on this

randradas commented 2 years ago

Hello @karezza I need further investigation so I'll create an internal task for it in our backlog.

karezza commented 2 years ago

Sounds like a plan. Were you able to reproduce the issue?

On Tue, Oct 5, 2021, 3:37 AM Roberto Andradas @.***> wrote:

Hello @karezza https://github.com/karezza I need further investigation so I'll create an internal task for it in our backlog.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bitnami/charts/issues/7536#issuecomment-934239805, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEHXQ7MUB6PBGC6MYVZR2P3UFLBMLANCNFSM5EKVHHDA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

randradas commented 2 years ago

Not sure yet. I'm not pretty sure if I have a proper cacert.pem file.

would it be ok to get it from here?

karezza commented 2 years ago

I'm not sure about your test, in my case I'm using the public key of the ActiveDirectory certificate authority which can be obtained in this way: https://docs.microsoft.com/en-us/troubleshoot/windows-server/identity/export-root-certification-authority-certificate

lknite commented 2 years ago

Without an AD environment to test with, I would think you'd have to verify in the code that the same method being used to test the connection with the ldaps server is also being used just below when testing the user account to verify it can perform a lookup in ad. By the same code I mean there must be a section in both which is using the auth.tls.existingSecret. I'm guessing it's not enabled in the second one by mistake, if it is, then there must be a bug in the way the auth.tls is getting added to the keystore/truststore. Or maybe I'm doing something wrong, I have the same value I am manually loading into the keystore in my docker container added to both the keystore/truststore values. I can't imagine I'm the first to use this chart in an AD environment ...

randradas commented 2 years ago

Thank you @lknite I will use the same cacert.pem file, couldn't reproduce it yet. Working on it.

lknite commented 2 years ago

I noticed there is an https-keystore.jks now, is that new? If so, could there have been a missed change associated with it?

lknite commented 2 years ago

It occured to me that I might need to set a password for the truststore/keystore, even though I was using the default of 'changeit'. I specified the password like so:

# required otherwise you get a blank page when visiting the admin console
proxyAddressForwarding: true

# to specify the ad ca public certificate in order to access ldaps
auth:
  tls:
    enabled: true
    existingSecret: keycloak-certs
    truststorePassword: changeit

I noticed this change was reflected in the secret 'keycloak' as:

Data
====
management-password:     10 bytes
tls-truestore-password:  8 bytes
admin-password:          10 bytes

Looks like there may be a typo, 'tls-truestore-password', which probably should be 'tls-truststore-password'?. This also exists in the values.yaml file, if this is a typo maybe this causing things not to work?

lknite commented 2 years ago

I checked inside the container and I can see the following environment variables being set, which look right:

KEYCLOAK_TLS_TRUSTSTORE_FILE=/opt/bitnami/keycloak/certs/keycloak.truststore.jks
KEYCLOAK_ENABLE_TLS=true
KEYCLOAK_TLS_TRUSTSTORE_PASSWORD=changeit
KEYCLOAK_TLS_CREATE_KEYSTORE=false
KEYCLOAK_TLS_KEYSTORE_FILE=/opt/bitnami/keycloak/certs/keycloak.keystore.jks

What to check next?

lknite commented 2 years ago

This worked after I also set the keystore password, so for ldaps it must use the keystore and not the truststore:

auth:
  tls:
    enabled: true
    existingSecret: keycloak-certs
    keystorePassword: changeit
    truststorePassword: changeit
randradas commented 2 years ago

Great @lknite I'm happy you made it, to be honest I was not being able to make it work either. It is supposed that keystore is for your application credentials and truststore for credentials of the others you trust.

Nold360 commented 2 years ago

The original keycloak image has a simple init script which generates the Truststore. And you can simply add a cert by using the env X509_CA_BUNDLE=/path/to/my/ca.crt. Why is this so complicated here, i'm trying for more then half a day by now >.< ....

See: https://github.com/keycloak/keycloak-containers/blob/main/server/tools/x509.sh

migruiz4 commented 2 years ago

Hi @Nold360,

Sharing my thoughts about this issue:

We are conditionally setting the env variables for the Keystore password and Truststore password: https://github.com/bitnami/charts/blob/e67be9af57fece432bd20df8cf278865cad3768e/bitnami/keycloak/templates/statefulset.yaml#L126-L149

By that I mean, that if Keystore or Truststore passwords are not provided, the chart won't complain about it. Neither will the container logic.

This section is the logic used by the container to configure TLS settings: https://github.com/bitnami/bitnami-docker-keycloak/blob/bc22bfdb1bc775f0cab80ee4faf95fefc9281495/16/debian-10/rootfs/opt/bitnami/scripts/libkeycloak.sh#L239-L258

The underlying jboss script is very similar to the logic used by our image: https://github.com/keycloak/keycloak-containers/blob/main/server/tools/cli/x509-keystore.cli

The main difference relies on the Keycloak image autogenerating the certificates, while our chart requires users to create their own certificates and store them in keystore/truststore.

The instructions about how to generate the certificates and secret can be found here at values.yaml: https://github.com/bitnami/charts/blob/09013945ecc643fbff35e4a4c6f4e99d19514228/bitnami/keycloak/values.yaml#L140-L148

But if you have any question or find any issues, please let us know.

Nold360 commented 2 years ago

Hi @migruiz4 Thanks for your reply. I get that it's not impossible to add new certificates. It just doesn't fit well in my GitOps workflow.

When i have to create a secret first, it also means i need to generate a truststore password. But I don't want or need to know that password. Also I have to generate that truststore somehow & store it in a secret.

That makes reproducing the whole workflow kinda complicated vs. just adding the cert to my chart values. Can't we add something like https://github.com/keycloak/keycloak-containers/blob/main/server/tools/x509.sh ? Maybe to the image entrypoint or as an init container?

I could work on a PR for that.

migruiz4 commented 2 years ago

Since it is a QoL feature for the chart, and we already had the code to generate the Keystore/Truststore given a TLS secret for autoGenerated certs, I have submitted this PR https://github.com/bitnami/charts/pull/9039 that would allow users to provide PEM certificates, which I think fits your case.

Unfortunately, it is still necessary to provide both the Node certs and the CA certs, because our container and chart expect both the Keystore and Truststore to be provided together. Making those changes would require more significant changes to both the Keycloak container and chart.

If you would like to give it a try and contribute to those changes, we will be very happy to review them and help with anything if needed. I have created a task to revisit this in the future, but I can not give you an ETA for when a TLS rework for Keycloak will be addressed.