Cannot start when using provider in Azure Container Apps instance

[brutaldev]

This seems to work perfectly when used locally in Docker Desktop, but when the providers are deployed to Azure, the container cannot start and fails with the following exception:

Exception in thread "main" java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:118)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:62)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:33)
Caused by: java.util.ServiceConfigurationError: org.eclipse.microprofile.config.spi.ConfigSourceProvider: Provider org.keycloak.quarkus.runtime.configuration.KeycloakConfigSourceProvider could not be instantiated
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586)
at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813)
at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729)
at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)
at io.smallrye.config.SmallRyeConfigBuilder.discoverSources(SmallRyeConfigBuilder.java:130)
at io.smallrye.config.SmallRyeConfig$ConfigSources.buildSources(SmallRyeConfig.java:764)
at io.smallrye.config.SmallRyeConfig$ConfigSources.<init>(SmallRyeConfig.java:715)
at io.smallrye.config.SmallRyeConfig.<init>(SmallRyeConfig.java:81)
at io.smallrye.config.SmallRyeConfigBuilder.build(SmallRyeConfigBuilder.java:724)
at io.quarkus.runtime.configuration.QuarkusConfigFactory.getConfigFor(QuarkusConfigFactory.java:27)
at io.smallrye.config.SmallRyeConfigProviderResolver.getConfig(SmallRyeConfigProviderResolver.java:78)
at io.smallrye.config.SmallRyeConfigProviderResolver.getConfig(SmallRyeConfigProviderResolver.java:66)
at org.keycloak.quarkus.runtime.configuration.Configuration.getConfig(Configuration.java:72)
at org.keycloak.quarkus.runtime.configuration.Configuration.getConfigValue(Configuration.java:124)
at org.keycloak.quarkus.runtime.cli.Picocli.validateConfig(Picocli.java:326)
at org.keycloak.quarkus.runtime.KeycloakMain.main(KeycloakMain.java:84)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
... 3 more
Caused by: java.lang.ExceptionInInitializerError
at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1160)
at java.base/jdk.internal.reflect.MethodHandleAccessorFactory.ensureClassInitialized(MethodHandleAccessorFactory.java:300)
at java.base/jdk.internal.reflect.MethodHandleAccessorFactory.newConstructorAccessor(MethodHandleAccessorFactory.java:103)
at java.base/jdk.internal.reflect.ReflectionFactory.newConstructorAccessor(ReflectionFactory.java:200)
at java.base/java.lang.reflect.Constructor.acquireConstructorAccessor(Constructor.java:549)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789)
... 18 more
Caused by: java.lang.RuntimeException: Failed to open /opt/keycloak/lib/../providers/io.phasetwo.keycloak-keycloak-events-0.32.jar
at io.quarkus.bootstrap.runner.JarResource.ensureJarFileIsOpen(JarResource.java:172)
at io.quarkus.bootstrap.runner.JarResource.readLockAcquireAndGetJarReference(JarResource.java:153)
at io.quarkus.bootstrap.runner.JarResource.getResourceURL(JarResource.java:101)
at io.quarkus.bootstrap.runner.RunnerClassLoader.findResources(RunnerClassLoader.java:214)
at java.base/java.lang.ClassLoader.getResources(ClassLoader.java:1483)
at io.smallrye.common.classloader.ClassPathUtils.consumeAsPaths(ClassPathUtils.java:84)
at io.smallrye.config.AbstractLocationConfigSourceLoader.tryClassPath(AbstractLocationConfigSourceLoader.java:141)
at org.keycloak.quarkus.runtime.configuration.KeycloakPropertiesConfigSource$InClassPath.tryClassPath(KeycloakPropertiesConfigSource.java:79)
at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:104)
at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:87)
... 39 more

Have you seen this before?

This is a custom build of Keycloak that copies the provider JAR files before building the image:

...
ADD --chown=keycloak:keycloak --chmod=644 https://repo1.maven.org/maven2/io/phasetwo/keycloak/keycloak-orgs/0.78/keycloak-orgs-0.78.jar /opt/keycloak/providers/io.phasetwo.keycloak-keycloak-orgs-0.78.jar
ADD --chown=keycloak:keycloak --chmod=644 https://repo1.maven.org/maven2/io/phasetwo/keycloak/keycloak-events/0.32/keycloak-events-0.32.jar /opt/keycloak/providers/io.phasetwo.keycloak-keycloak-events-0.32.jar
RUN /opt/keycloak/bin/kc.sh build

FROM quay.io/keycloak/keycloak:25.0.6-0
COPY --from=builder /opt/keycloak/ /opt/keycloak/
...

Using the environment variables WEBHOOK_URI and WEBHOOK_SECRET.

[xgp]

I haven't, but my guess is that 644 is not sufficient. Try it with 777 and then back off from there.

Also, FYI, if you are trying to include keycloak-orgs like that, it won't work. It requires a set of external dependencies that are not included in that jar. Take a look in the phasetwo-containers repo for information on how our image gets built.

[brutaldev]

Thanks, I'll take a look at how your containers are built.

keycloak-orgs is a dependency of keycloak-events so that'y why I include it as well. Odd that this works locally but not when deployed.

I'll try with 777 permissions and include all other dependencies and let you know how far I get.

[xgp]

keycloak-orgs is a dependency of keycloak-events

No. It is not a runtime dependency. We only use it in tests https://github.com/p2-inc/keycloak-events/blob/main/pom.xml#L267-L271

[brutaldev]

Found the issue. The JAR file was not downloaded correctly. I found this extra exception in the logs that indicated there was something wrong with the file itself: Caused by: java.util.zip.ZipException: zip END header not found

Thanks again for the help, this is working as expected now.