quarkus-bot[bot]
commented
1 week ago /cc @cescoffier (rest-client), @geoand (rest-client)
Open Lainiel opened 1 week ago
quarkus-bot[bot]
commented
1 week ago /cc @cescoffier (rest-client), @geoand (rest-client)
geoand
commented
1 week ago What version of Quarkus are you using?
Asking because the prefered way since a couple of versions ago to set TLS related configuration is via quarkus.rest-client.tls-configuration-name. See this for more details.
Lainiel
commented
1 week ago The quarkus version is 3.15.1 I did try using the tls-configuration-name property already, but the issue was still the same.
Edit: I retested it just to be sure with these settings:
quarkus.tls.custom-cert.trust-store.jks.path=/etc/config/truststore/truststore.ts
quarkus.tls.custom-cert.trust-store.jks.password=redacted
quarkus.rest-client.custom-api.tls-configuration-name=custom-certIt still doesn't work. The error is the same.
cescoffier
commented
1 week ago Did you enable alpn? I saw an intriguing message about that.
Lainiel
commented
1 week ago I did not change any alpn settings in my configuration.
But I tried disabling it with the following configuration and it did not help. The issue persists.
quarkus.tls.custom-cert.trust-store.jks.path=/etc/config/truststore/truststore.ts
quarkus.tls.custom-cert.trust-store.jks.password=redacted
quarkus.tls.custom-cert.alpn=false
quarkus.rest-client.custom-api.tls-configuration-name=custom-cert
quarkus.rest-client.custom-api.alpn=false
I would have enabled it actually.
cescoffier
commented
1 week ago BTW, that could be a server issue, as it seems we are waiting for a message from the server.
Lainiel
commented
1 week ago Enabling them didn't help either. It is not a server issue, because as I said in my original post it works when I set the certificate using system properties.
geoand
commented
6 days ago Is there any debugging info or network capture info that could point to what is different between the working and non-working versions?
Lainiel
commented
1 day ago I don't have much debugging info to provide. I've compared the ClientHello messages from the working version and the non-working version and they're pretty much the same.
The only differences are the random, session id and key_exchange fields.
Here is a client hello that works:
"ClientHello": {
"client version" : "TLSv1.2",
"random" : "419717CF477833FB8791DC6FC5785B1A117B5139D91E3319C5C3A801B7041BD1",
"session id" : "9CE0D079DE49020F593CD0E1337E285C36158D64403570AC9AE3CDD2C5422FF7",
"cipher suites" : "[TLS_AES_256_GCM_SHA384(0x1302), TLS_AES_128_GCM_SHA256(0x1301), TLS_CHACHA20_POLY1305_SHA256(0x1303), TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA9), TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA8), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_DHE_RSA_WITH_AES_256_GCM_SHA384(0x009F), TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCAA), TLS_DHE_RSA_WITH_AES_128_GCM_SHA256(0x009E), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_DHE_RSA_WITH_AES_256_CBC_SHA256(0x006B), TLS_DHE_RSA_WITH_AES_128_CBC_SHA256(0x0067), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA(0xC00A), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA(0xC014), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA(0xC009), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA(0xC013), TLS_DHE_RSA_WITH_AES_256_CBC_SHA(0x0039), TLS_DHE_RSA_WITH_AES_128_CBC_SHA(0x0033), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_RSA_WITH_AES_128_GCM_SHA256(0x009C), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_RSA_WITH_AES_256_CBC_SHA(0x0035), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_EMPTY_RENEGOTIATION_INFO_SCSV(0x00FF)]",
"compression methods" : "00",
"extensions" : [
"server_name (0)": {
type=host_name (0), value=redacted
},
"status_request (5)": {
"certificate status type": ocsp
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
},
"supported_groups (10)": {
"versions": [x25519, secp256r1, secp384r1, secp521r1, x448, ffdhe2048, ffdhe3072, ffdhe4096, ffdhe6144, ffdhe8192]
},
"ec_point_formats (11)": {
"formats": [uncompressed]
},
"status_request_v2 (17)": {
"cert status request": {
"certificate status type": ocsp_multi
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
}
},
"extended_master_secret (23)": {
<empty>
},
"session_ticket (35)": {
<empty>
},
"signature_algorithms (13)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, ed25519, ed448, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, ecdsa_sha224, rsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1]
},
"supported_versions (43)": {
"versions": [TLSv1.3, TLSv1.2]
},
"psk_key_exchange_modes (45)": {
"ke_modes": [psk_dhe_ke]
},
"signature_algorithms_cert (50)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, ed25519, ed448, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, ecdsa_sha224, rsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1]
},
"key_share (51)": {
"client_shares": [
{
"named group": x25519
"key_exchange": {
0000: 89 06 1A 41 EB B6 49 45 BC C9 91 61 C5 23 E4 D1 ...A..IE...a.#..
0010: D6 6F A0 4C 99 38 A6 4E F5 82 25 AD 67 73 27 4F .o.L.8.N..%.gs'O
}
},
{
"named group": secp256r1
"key_exchange": {
0000: 04 E4 05 D7 F2 B8 34 06 D4 98 0C E4 23 BE 7D CB ......4.....#...
0010: A3 2C 36 26 E4 AF 77 3C 19 89 F4 DB 13 49 82 03 .,6&..w<.....I..
0020: 32 19 2D 93 26 0C 14 DF C2 7D E1 BD EA 37 A4 F0 2.-.&........7..
0030: F8 52 00 36 FC 1C E2 74 4A C8 1F 6D 4F 7C 6E 86 .R.6...tJ..mO.n.
0040: 77
}
},
]
}
]
}Here is the one that doesn't and never receives ServerHello:
"ClientHello": {
"client version" : "TLSv1.2",
"random" : "E80CD6A0836C9D2DA789FB0D105D12E4E57ABBED2C868DD7310265583C91BEFD",
"session id" : "C372103A242A1248AB24F4A9276E771CEAF0ADF996F0EE764B4B8F84DD8F561F",
"cipher suites" : "[TLS_AES_256_GCM_SHA384(0x1302), TLS_AES_128_GCM_SHA256(0x1301), TLS_CHACHA20_POLY1305_SHA256(0x1303), TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA9), TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA8), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_DHE_RSA_WITH_AES_256_GCM_SHA384(0x009F), TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCAA), TLS_DHE_RSA_WITH_AES_128_GCM_SHA256(0x009E), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_DHE_RSA_WITH_AES_256_CBC_SHA256(0x006B), TLS_DHE_RSA_WITH_AES_128_CBC_SHA256(0x0067), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA(0xC00A), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA(0xC014), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA(0xC009), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA(0xC013), TLS_DHE_RSA_WITH_AES_256_CBC_SHA(0x0039), TLS_DHE_RSA_WITH_AES_128_CBC_SHA(0x0033), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_RSA_WITH_AES_128_GCM_SHA256(0x009C), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_RSA_WITH_AES_256_CBC_SHA(0x0035), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_EMPTY_RENEGOTIATION_INFO_SCSV(0x00FF)]",
"compression methods" : "00",
"extensions" : [
"server_name (0)": {
type=host_name (0), value=redacted
},
"status_request (5)": {
"certificate status type": ocsp
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
},
"supported_groups (10)": {
"versions": [x25519, secp256r1, secp384r1, secp521r1, x448, ffdhe2048, ffdhe3072, ffdhe4096, ffdhe6144, ffdhe8192]
},
"ec_point_formats (11)": {
"formats": [uncompressed]
},
"status_request_v2 (17)": {
"cert status request": {
"certificate status type": ocsp_multi
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
}
},
"extended_master_secret (23)": {
<empty>
},
"session_ticket (35)": {
<empty>
},
"signature_algorithms (13)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, ed25519, ed448, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, ecdsa_sha224, rsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1]
},
"supported_versions (43)": {
"versions": [TLSv1.3, TLSv1.2]
},
"psk_key_exchange_modes (45)": {
"ke_modes": [psk_dhe_ke]
},
"signature_algorithms_cert (50)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, ed25519, ed448, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, ecdsa_sha224, rsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1]
},
"key_share (51)": {
"client_shares": [
{
"named group": x25519
"key_exchange": {
0000: 34 3B A2 F3 AF AD D3 4B 06 D4 A2 E9 2B E2 BA 6C 4;.....K....+..l
0010: 0C 57 DE 6C FA 58 43 84 06 5A 38 49 71 F3 59 27 .W.l.XC..Z8Iq.Y'
}
},
{
"named group": secp256r1
"key_exchange": {
0000: 04 E0 29 D8 F7 E6 89 3A 4E 9A 65 69 78 88 E8 19 ..)....:N.eix...
0010: F9 7D B1 4F FB B0 DE 7F 94 B9 C7 97 69 94 42 BB ...O........i.B.
0020: A2 9F 75 6D B4 82 26 7B 26 A9 E3 61 0D 97 B6 ED ..um..&.&..a....
0030: F0 EC F8 B4 88 11 2F 8F E1 A3 04 DA A5 04 2B C4 ....../.......+.
0040: 08
}
},
]
}
]
}
benDeMtp
commented
1 day ago Hi, we met the same behavior randomly. But it's happened only with native images and only on openshift :/ After several restarts, it works as expected.
io.quarkus.runtime.ApplicationLifecycleManager run
ERROR: Failed to start application (with profile [prod])
java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
at io.quarkus.runner.GeneratedMain.main(Unknown Source)
at java.base@21.0.3/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
Caused by: java.lang.RuntimeException: An error occurred while attempting to fetch configuration from Consul.
at io.quarkus.consul.config.runtime.ConsulConfigSourceFactory.getConfigSources(ConsulConfigSourceFactory.java:79)
at io.quarkus.consul.config.runtime.ConsulConfigSourceFactory.getConfigSources(ConsulConfigSourceFactory.java:28)
at io.quarkus.consul.config.runtime.ConsulConfigSourceFactory.getConfigSources(ConsulConfigSourceFactory.java:19)
at io.smallrye.config.ConfigSourceFactory$ConfigurableConfigSourceFactory.getConfigSources(ConfigSourceFactory.java:58)
at io.smallrye.config.ConfigurableConfigSource.getConfigSources(ConfigurableConfigSource.java:50)
at io.smallrye.config.SmallRyeConfig$ConfigSources.mapLateSources(SmallRyeConfig.java:669)
at io.smallrye.config.SmallRyeConfig$ConfigSources.<init>(SmallRyeConfig.java:553)
at io.smallrye.config.SmallRyeConfig.<init>(SmallRyeConfig.java:69)
at io.smallrye.config.SmallRyeConfigBuilder.build(SmallRyeConfigBuilder.java:629)
at io.quarkus.runtime.generated.Config.readConfig(Unknown Source)
at io.quarkus.deployment.steps.RuntimeConfigSetup.deploy(Unknown Source)
... 7 more
Caused by: java.util.concurrent.CompletionException: javax.net.ssl.SSLHandshakeException: Failed to create SSL connection
at io.smallrye.mutiny.operators.uni.UniBlockingAwait.await(UniBlockingAwait.java:79)
at io.smallrye.mutiny.groups.UniAwait.atMost(UniAwait.java:65)
at io.quarkus.consul.config.runtime.ConsulConfigSourceFactory.getConfigSources(ConsulConfigSourceFactory.java:76)
... 17 more
Caused by: javax.net.ssl.SSLHandshakeException: Failed to create SSL connection
at io.vertx.core.net.impl.ChannelProvider$1.userEventTriggered(ChannelProvider.java:127)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376)
at io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368)
at io.netty.handler.ssl.SslUtils.handleHandshakeFailure(SslUtils.java:492)
at io.netty.handler.ssl.SslHandler$7.run(SslHandler.java:2221)
at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base@21.0.3/java.lang.Thread.runWith(Thread.java:1596)
at java.base@21.0.3/java.lang.Thread.run(Thread.java:1583)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:896)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:872)
Caused by: io.netty.handler.ssl.SslHandshakeTimeoutException: handshake timed out after 10000ms
at io.netty.handler.ssl.SslHandler$7.run(SslHandler.java:2217)
... 13 moreThe consul service was active and available at this time.
We removed consul request, and the call of the next service in the chain ( Dynamics) cause the same issue.
Quarkus version 3.8.3
Lainiel
commented
2 hours ago To clarify we are also running the application in OpenShift.
Describe the bug
In our application we have a rest-client that needs a different truststore certificate for the rest-client a oidc-client.
The truststore needed for the oidc-client is a default java truststore. The certificate for the rest-client is a custom truststore.
When the setting in application properties are as following:
I get a problem when calling the service. The OidcClient is able to acquire the tokens, but the rest-client creates a timeout during handshake. It sends a ClientHello handshake message, but timeouts while waiting for the ServerHello handshake message.
When I use the non-reactive rest-client the request goes through fine.
I've also figured out a workaround for the reactive rest-client, because when I set the rest-client truststore using system properties like this:
And then I reset the certificate for the oidc-client using these settings:
Then it works and no handshake timeout happens.
Expected behavior
No timeout during handshake with custom rest-client truststore.
Actual behavior
How to Reproduce?
No response
Output of
uname -aorverNo response
Output of
java -versionNo response
Quarkus version or git rev
No response
Build tool (ie. output of
mvnw --versionorgradlew --version)No response
Additional information
No response