We should to use our enterprise keycloak over https. The keycloak https endpoint is signed by internal PKI.
On operator v1, I doesn't found the way to mount keystore on pod with our internal pki certificate ?
Without this, we get ssl error to validate keycloak certificat.
The logs about this issue:
" src_ip="10.53.20.2:45582" x_forwarded_for="10.221.123.9" path="/apis/registry/v2/users/me" response_code="500" error_msg="OIDC server is not available at the 'https://keycloak-rec.hm.dm.ad/auth/realms/kafka_hpd1' URL. Please make sure it is correct. Note it has to end with a realm value if you work with Keycloak, for example: 'https://localhost:8180/auth/realms/quarkus'" method="GET"
2023-06-20 15:03:24 ERROR <> [io.quarkus.vertx.http.runtime.QuarkusErrorHandler] (vert.x-eventloop-thread-1) HTTP Request to /apis/registry/v2/users/me failed, error id: 92932325-9320-44dd-a3a7-d9c877d90404-2: io.quarkus.oidc.OIDCException: OIDC server is not available at the 'https://keycloak-rec.hm.dm.ad/auth/realms/kafka_hpd1' URL. Please make sure it is correct. Note it has to end with a realm value if you work with Keycloak, for example: 'https://localhost:8180/auth/realms/quarkus'
at io.quarkus.oidc.runtime.OidcRecorder.toOidcException(OidcRecorder.java:232)
at io.quarkus.oidc.runtime.OidcRecorder$5.apply(OidcRecorder.java:301)
at io.quarkus.oidc.runtime.OidcRecorder$5.apply(OidcRecorder.java:295)
at io.smallrye.context.impl.wrappers.SlowContextualBiFunction.apply(SlowContextualBiFunction.java:21)
at io.smallrye.mutiny.operators.uni.UniOnItemOrFailureFlatMap$UniOnItemOrFailureFlatMapProcessor.performInnerSubscription(UniOnItemOrFailureFlatMap.java:86)
at io.smallrye.mutiny.operators.uni.UniOnItemOrFailureFlatMap$UniOnItemOrFailureFlatMapProcessor.onFailure(UniOnItemOrFailureFlatMap.java:65)
at io.smallrye.mutiny.operators.uni.UniOperatorProcessor.onFailure(UniOperatorProcessor.java:54)
at io.smallrye.mutiny.operators.uni.UniOnFailureTransform$UniOnFailureTransformProcessor.onFailure(UniOnFailureTransform.java:64)
at io.smallrye.mutiny.operators.uni.builders.UniCreateFromPublisher$PublisherSubscriber.onError(UniCreateFromPublisher.java:81)
at io.smallrye.mutiny.helpers.HalfSerializer.onError(HalfSerializer.java:56)
at io.smallrye.mutiny.helpers.StrictMultiSubscriber.onFailure(StrictMultiSubscriber.java:91)
at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
at io.smallrye.mutiny.subscription.SerializedSubscriber.onFailure(SerializedSubscriber.java:102)
at io.smallrye.mutiny.operators.multi.MultiRetryWhenOp$RetryWhenOperator.testOnFailurePredicate(MultiRetryWhenOp.java:136)
at io.smallrye.mutiny.operators.multi.MultiRetryWhenOp$RetryWhenOperator.onFailure(MultiRetryWhenOp.java:119)
at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
at io.smallrye.mutiny.converters.uni.UniToMultiPublisher$UniToMultiSubscription.onFailure(UniToMultiPublisher.java:103)
at io.smallrye.mutiny.operators.uni.UniOperatorProcessor.onFailure(UniOperatorProcessor.java:54)
at io.smallrye.mutiny.vertx.AsyncResultUni.lambda$subscribe$1(AsyncResultUni.java:37)
at io.vertx.mutiny.ext.web.client.HttpRequest$10.handle(HttpRequest.java:738)
at io.vertx.mutiny.ext.web.client.HttpRequest$10.handle(HttpRequest.java:733)
at io.vertx.ext.web.client.impl.HttpContext.handleFailure(HttpContext.java:396)
at io.vertx.ext.web.client.impl.HttpContext.execute(HttpContext.java:390)
at io.vertx.ext.web.client.impl.HttpContext.next(HttpContext.java:365)
at io.vertx.ext.web.client.impl.HttpContext.fire(HttpContext.java:332)
at io.vertx.ext.web.client.impl.HttpContext.fail(HttpContext.java:313)
at io.vertx.ext.web.client.impl.HttpContext.lambda$handleCreateRequest$7(HttpContext.java:536)
at io.vertx.core.impl.future.FutureImpl$3.onFailure(FutureImpl.java:153)
at io.vertx.core.impl.future.FutureBase.lambda$emitFailure$1(FutureBase.java:69)
at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:81)
at io.vertx.core.impl.DuplicatedContext.execute(DuplicatedContext.java:173)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:66)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:230)
at io.vertx.core.impl.future.PromiseImpl.tryFail(PromiseImpl.java:23)
at io.vertx.core.http.impl.HttpClientImpl.lambda$doRequest$8(HttpClientImpl.java:659)
at io.vertx.core.net.impl.pool.Endpoint.lambda$getConnection$0(Endpoint.java:52)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:162)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:123)
at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:50)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:274)
at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:22)
at io.vertx.core.net.impl.pool.SimpleConnectionPool$ConnectFailed$1.run(SimpleConnectionPool.java:382)
at io.vertx.core.net.impl.pool.CombinerExecutor.submit(CombinerExecutor.java:50)
at io.vertx.core.net.impl.pool.SimpleConnectionPool.execute(SimpleConnectionPool.java:245)
at io.vertx.core.net.impl.pool.SimpleConnectionPool.lambda$connect$2(SimpleConnectionPool.java:259)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint.lambda$connect$2(SharedClientHttpStreamEndpoint.java:104)
at io.vertx.core.impl.future.FutureImpl$3.onFailure(FutureImpl.java:153)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:75)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:230)
at io.vertx.core.impl.future.Composition$1.onFailure(Composition.java:66)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:75)
at io.vertx.core.impl.future.FailedFuture.addListener(FailedFuture.java:98)
at io.vertx.core.impl.future.Composition.onFailure(Composition.java:55)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:75)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:230)
at io.vertx.core.impl.future.PromiseImpl.tryFail(PromiseImpl.java:23)
at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:50)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:274)
at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:22)
at io.vertx.core.net.impl.NetClientImpl.failed(NetClientImpl.java:294)
at io.vertx.core.net.impl.NetClientImpl.lambda$connectInternal$4(NetClientImpl.java:262)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:109)
at io.vertx.core.net.impl.ChannelProvider$1.userEventTriggered(ChannelProvider.java:131)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:324)
at io.netty.handler.ssl.SslHandler.handleUnwrapThrowable(SslHandler.java:1259)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1240)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:360)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:303)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:298)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1076)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1063)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1010)
at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1548)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1394)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1235)
... 20 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:276)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:632)
... 31 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:146)
at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:127)
at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
... 37 more
2023-06-20 15:03:24 INFO <> [io.apicurio.common.apps.logging.audit.AuditLogService] (vert.x-eventloop-thread-1) registry.audit action="authenticate" result="failure" src_ip="10.53.20.2:45582" x_forwarded_for="10.221.123.9" path="/apis/registry/v2/users/me" response_code="500" error_msg="OIDC server is not available at the 'https://keycloak-rec.hm.dm.ad/auth/realms/kafka_hpd1' URL. Please make sure it is correct. Note it has to end with a realm value if you work with Keycloak, for example: 'https://localhost:8180/auth/realms/quarkus'" method="GET"
Hi,
We should to use our enterprise keycloak over https. The keycloak https endpoint is signed by internal PKI.
On operator v1, I doesn't found the way to mount keystore on pod with our internal pki certificate ? Without this, we get ssl error to validate keycloak certificat.
The logs about this issue:
Best regards, SL