search

microcks / microcks-ansible-operator

Kubernetes Operator for easy setup and management of Microcks installs
https://microcks.io
Apache License 2.0
26 stars 6 forks source link

Error 500 on /api/jobs/ endpoints caused by CertPathValidatorException #96

Closed PauAL closed 1 year ago

PauAL commented 1 year ago

Describe the bug

We installed microcks using microcks-ansible-operator (1.7.0 version, Build timestamp 2023-03-08T15:54:54Z) on openshift using this configuration:

apiVersion: microcks.github.io/v1alpha1
kind: MicrocksInstall
spec:
  keycloak:
    install: true
  microcks:
    replicas: 1
  mongodb:
    install: true
    persistent: true
    volumeSize: 2Gi
  name: my-microcksinstall
  postman:
    replicas: 1
  version: 1.7.0

Installation worked and after successfully uploading a dummy artifact we were not able to see it in "APIs | Services" page. I checked and saw some 500 errors, all of them where from request to “/api/jobs/*” endpoints. In my-microcksinstall logs I could see the following trace:

09:26:37.565 [WARN] org.keycloak.adapters.KeycloakDeployment - Failed to load URLs from https://my-microcksinstall-keycloak-###/realms/microcks/.well-known/openid-configuration
javax.net.ssl.SSLHandshakeException: Certificates do not conform to algorithm constraints
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:371)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:309)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1357)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1232)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1175)
    at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:480)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:458)
    at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:201)
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1510)
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1425)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426)
    at org.apache.http.conn.ssl.SSLSocketFactory.createLayeredSocket(SSLSocketFactory.java:570)
    at org.keycloak.adapters.SniSSLSocketFactory.createLayeredSocket(SniSSLSocketFactory.java:119)
    at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:554)
    at org.keycloak.adapters.SniSSLSocketFactory.connectSocket(SniSSLSocketFactory.java:114)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
    at org.keycloak.adapters.KeycloakDeployment.getOidcConfiguration(KeycloakDeployment.java:230)
    at org.keycloak.adapters.KeycloakDeployment.resolveUrls(KeycloakDeployment.java:182)
    at org.keycloak.adapters.KeycloakDeployment.getJwksUrl(KeycloakDeployment.java:281)
    at org.keycloak.adapters.rotation.JWKPublicKeyLocator.sendRequest(JWKPublicKeyLocator.java:98)
    at org.keycloak.adapters.rotation.JWKPublicKeyLocator.getPublicKey(JWKPublicKeyLocator.java:63)
    at org.keycloak.adapters.rotation.AdapterTokenVerifier.getPublicKey(AdapterTokenVerifier.java:121)
    at org.keycloak.adapters.rotation.AdapterTokenVerifier.createVerifier(AdapterTokenVerifier.java:111)
    at org.keycloak.adapters.rotation.AdapterTokenVerifier.verifyToken(AdapterTokenVerifier.java:47)
    at org.keycloak.adapters.BearerTokenRequestAuthenticator.authenticateToken(BearerTokenRequestAuthenticator.java:105)
    at org.keycloak.adapters.BearerTokenRequestAuthenticator.authenticate(BearerTokenRequestAuthenticator.java:90)
    at org.keycloak.adapters.RequestAuthenticator.authenticate(RequestAuthenticator.java:67)
    at org.keycloak.adapters.tomcat.AbstractKeycloakAuthenticatorValve.authenticateInternal(AbstractKeycloakAuthenticatorValve.java:203)
    at org.keycloak.adapters.tomcat.KeycloakAuthenticatorValve.authenticate(KeycloakAuthenticatorValve.java:50)
    at org.keycloak.adapters.tomcat.KeycloakAuthenticatorValve.doAuthenticate(KeycloakAuthenticatorValve.java:57)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:625)
    at org.keycloak.adapters.tomcat.AbstractKeycloakAuthenticatorValve.invoke(AbstractKeycloakAuthenticatorValve.java:181)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
    at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:769)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:891)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1784)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.security.cert.CertificateException: Certificates do not conform to algorithm constraints
    at java.base/sun.security.ssl.AbstractTrustManagerWrapper.checkAlgorithmConstraints(SSLContextImpl.java:1573)
    at java.base/sun.security.ssl.AbstractTrustManagerWrapper.checkAdditionalTrust(SSLContextImpl.java:1498)
    at java.base/sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:1442)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1341)
    ... 56 common frames omitted
Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: SHA1withRSA
    at java.base/sun.security.provider.certpath.AlgorithmChecker.check(AlgorithmChecker.java:237)
    at java.base/sun.security.ssl.AbstractTrustManagerWrapper.checkAlgorithmConstraints(SSLContextImpl.java:1569)
    ... 59 common frames omitted
09:26:37.567 [ERROR] org.apache.catalina.core.ContainerBase.[Tomcat].[localhost] - Exception Processing /api/services/map
java.lang.NullPointerException: Cannot invoke "String.length()" because "this.input" is null
    at java.base/java.net.URI$Parser.parse(URI.java:3165)
    at java.base/java.net.URI.<init>(URI.java:623)
    at java.base/java.net.URI.create(URI.java:904)
    at org.apache.http.client.methods.HttpGet.<init>(HttpGet.java:66)
    at org.keycloak.adapters.rotation.JWKPublicKeyLocator.sendRequest(JWKPublicKeyLocator.java:98)
    at org.keycloak.adapters.rotation.JWKPublicKeyLocator.getPublicKey(JWKPublicKeyLocator.java:63)
    at org.keycloak.adapters.rotation.AdapterTokenVerifier.getPublicKey(AdapterTokenVerifier.java:121)
    at org.keycloak.adapters.rotation.AdapterTokenVerifier.createVerifier(AdapterTokenVerifier.java:111)
    at org.keycloak.adapters.rotation.AdapterTokenVerifier.verifyToken(AdapterTokenVerifier.java:47)
    at org.keycloak.adapters.BearerTokenRequestAuthenticator.authenticateToken(BearerTokenRequestAuthenticator.java:105)
    at org.keycloak.adapters.BearerTokenRequestAuthenticator.authenticate(BearerTokenRequestAuthenticator.java:90)
    at org.keycloak.adapters.RequestAuthenticator.authenticate(RequestAuthenticator.java:67)
    at org.keycloak.adapters.tomcat.AbstractKeycloakAuthenticatorValve.authenticateInternal(AbstractKeycloakAuthenticatorValve.java:203)
    at org.keycloak.adapters.tomcat.KeycloakAuthenticatorValve.authenticate(KeycloakAuthenticatorValve.java:50)
    at org.keycloak.adapters.tomcat.KeycloakAuthenticatorValve.doAuthenticate(KeycloakAuthenticatorValve.java:57)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:625)
    at org.keycloak.adapters.tomcat.AbstractKeycloakAuthenticatorValve.invoke(AbstractKeycloakAuthenticatorValve.java:181)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
    at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:769)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:891)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1784)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:833)

It seems keycloak adapter is giving a NPE cause it is not able to retrieve any of the public certs published under “/protocol/openid-connect/certs”. This seems caused by the warning Caused by: java.security.cert.CertPathValidatorException: Algorithm constraints check failed on signature algorithm: SHA1withRSA.

Previous to this installation we installed microcks connected with our keycloak server. We got this error and checked again with operator installed keycloak just to check if it was caused by out keycloak configuration.

Expected behavior

No errors are expected.

Actual behavior

I receive a 500 {"timestamp":1681466461351,"status":500,"error":"Internal Server Error","path":"/api/jobs/count"} at every "/api/jobs". This error is blocking me to operate microks, I cannot see any of the APIs or services published.

How to Reproduce?

To reproduce this behaviour:

  1. Install microcks using configuration described.
  2. Navigate through microcks management web.
  3. Errors will appear microcks pod.

Microcks version or git rev

1.7.0 version, Build timestamp 2023-03-08T15:54:54Z

Install method (docker-compose, helm chart, operator, docker-desktop extension,...)

microcks-ansible-operator

Additional information

No response

lbroudoux commented 1 year ago

Thanks for raising this issue. I installed exact same version on my OpenShift cluster but I'm not able to reproduce it at the moment.

From what I understand, you're facing same issue with both Keycloak coming with Microcks and external Keycloak. Is that correct?

Are you cluster nodes running with some special constraints as the OS/System level? (thinking about SCC, SELinux profiles, seccomp profiles, FIPS enforcement or whatever....)

Also have you check the algorithm that are referenced when calling /realms/microcks/protocol/openid-connect/certs on you Keycloak server? On mine I just have RSA-OAEP and RS256. No Sha1 with RSA...

PauAL commented 1 year ago

Yes, I am facing the same issue with Microcks and with an external keycloak.

I would say we do not have any special security constraint as we where able to connect different apps to our keycloak already, but I will check it in the meantime just in case.

The certs algorithms are RSA-OAEP and RS256 (default certs configured in Microcks keycloak). I also do not know where this SHA1withRSA is coming from.

Regards.

lbroudoux commented 1 year ago

Too bad... Don't know how to get with this one, as I never faced this one... Looking for some Keycloak friends around... Maybe @M3lkior this would be something you're aware of?

M3lkior commented 1 year ago

hey ; nop; unfortunatelly ; maybe this is not a Keycloak issue.

you can maybe play with your key length that is maybe the root cause of your problem regarding this kind of post: https://stackoverflow.com/questions/21218217/ssl-handshake-exception-algorithm-constraints-check-failed-md5withrsa

PauAL commented 1 year ago

Hi, M3lkior, thank you for your comment. Unfortunately, I already checked and our key complies with the 1024+ bits long restriction so it should not be the problem.

I would appreciate any additional ideas to make this work.

Regards.

PauAL commented 1 year ago

Well, trying to figure it out I started reducing failure points. I configured in the operator keycloaks private URL to avoid the certification validation and it worked. Even when the certificate complies with the restriction there is something avoiding the connection.

Anyway, I would close this issue cause is seems it is not an operator problem, if you agree.