Keycloak doesn't start correctly on docker-compose up

[JochemBoersma]

Context: docker-compose up on my local machine (linux containers)

Following error occurs in bringing up the services: keycloak | 09:19:31,682 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([("subsystem" => "microprofile-metrics-smallrye")]): java.lang.NullPointerException

At restart, the following error occurs: keycloak | User with username 'admin' already added to '/opt/jboss/keycloak/standalone/configuration/keycloak-add-user.json'

Bringing the compose down, and removing all volumes before a restart doesn't solve. Any help or workaround would be appreciated.

[edwinmat]

This is a known KeyCloak issue that sometimes occur (We don't really understand yet when and why). Could you try the solution provided by the first answer in this stackexchange post: https://stackoverflow.com/questions/59599620/keycloak-8-user-with-username-admin-already-added.

[JochemBoersma]

Tried it according the stack-overflow solution, but received similar error. Both on my VM (Ubuntu 20.04) and on my laptop (Win10), and I get the same error..

From outside, in webbrowser, it returns this error: 503 - Service Unavailable

The pgadmin server is running (and accessable from outside) but empty.

Do you have an work-around? No problem for me to install the keycloak software outside the docker-stack, but wondering if this works anyways...

The error log:

keycloak    | 20:46:07,209 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([("subsystem" => "microprofile-metrics-smallrye")]): java.lang.NullPointerException
keycloak    |   at org.wildfly.extension.microprofile.metrics-smallrye@20.0.1.Final//org.wildfly.extension.microprofile.metrics.MicroProfileMetricsSubsystemAdd$2.execute(MicroProfileMetricsSubsystemAdd.java:86)
keycloak    |   at org.jboss.as.controller@12.0.3.Final//org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:999)
keycloak    |   at org.jboss.as.controller@12.0.3.Final//org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:743)
keycloak    |   at org.jboss.as.controller@12.0.3.Final//org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:467)
keycloak    |   at org.jboss.as.controller@12.0.3.Final//org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1413)
keycloak    |   at org.jboss.as.controller@12.0.3.Final//org.jboss.as.controller.ModelControllerImpl.boot(ModelControllerImpl.java:527)
keycloak    |   at org.jboss.as.controller@12.0.3.Final//org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:515)
keycloak    |   at org.jboss.as.controller@12.0.3.Final//org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:477)
keycloak    |   at org.jboss.as.server@12.0.3.Final//org.jboss.as.server.ServerService.boot(ServerService.java:451)
keycloak    |   at org.jboss.as.server@12.0.3.Final//org.jboss.as.server.ServerService.boot(ServerService.java:404)
keycloak    |   at org.jboss.as.controller@12.0.3.Final//org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:416)
keycloak    |   at java.base/java.lang.Thread.run(Thread.java:834)
keycloak    |
keycloak    | keycloak exited with code 1

[ewoudwerkman]

Strange error indeed.

What you could try is use a different keycloak image version. There have been a few bug releases after 11.0.0 which you setup might run into. You could change the docker-compose.yml in the BaseInfrastructure folder to use keycloak image version 11.0.3 instead of 11.0.0 and see if that helps.

After changing the docker-compose.yml, do a docker-compose pull to update the image, and then docker-compose up -d and have a look what the logs tell you.

[JochemBoersma]

I tried 11.0.3, without results, alas... Combined with system prune before, and performing the steps of the stack-overflow, it didn't succeed... Same errors as before. I also tried even newer docker images, but those returned other errors.

My docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:58:50 2021
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:10 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

and docker-compose version is:

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1g  21 Apr 2020

I really have no clue how to fix this....

[ewoudwerkman]

This is very frustrating. The whole thing about docker is that it should be easy to be reproducable...

Can you run the keycloak docker container on the command line? E.g. following the steps at the keycloak site (although this is a newer version than we use..., try also with the 11.0.0 version, too). I don't know if the configuration we copy into the keycloak container is still compatible with newer versions, but you could try that with the newest version. If you use an external keycloak, it requires some additional configuration, but it should work.

You could also try an earlier version than 11. When we first deployed this infrastructure, we were at version 6.01 of Keycloak.

If that doesn't work, please add the full log of keycloak when starting up. Maybe somewhere else things go bad.

[JochemBoersma]

reminds me of the following cartoon:

Next week I will give it another try, with your suggestions. Thanks for them anyway.

[edwinmat]

I think I found a possible cause... As you're probably using git for windows, git translates LFs to CRLFs by deafult on windows. This is okay for most of the files, but not the ones that are mounted into a docker container. This is the case for the file BaseInfrastructure/postgres/init-database.sh. If you add CRs to this file, bash will stop interpreting it at the first line already, with a message like "interpreter not found: /bin/bash^M". This script does a lot of initializations in the postgres database, like creating accounts, which is not happening anymore. One of the accounts that is not being created anymore is the keycloak account and therefore keycloak cannot properly start.

Like Ewoud already suggested, there's probably more errors in the logs (above the one that you pasted above). I'm pretty sure that there are errors that keycloak cannot connect to the database, or some issues with user keycloak.

What you could do is remove at least the postgres volume, make sure the init-database.sh file uses unix line-endings and restart from the beginning with the docker-compose up command. To change the CRLF back into LF, you can convert it with for example notepad++, or a dos2unix command line tool, or configure git in such a way that it doesn't automatically convert LF into CRLF (search for 'git autocrlf').

Please check and let us know