search

OpenIdentityPlatform / OpenDJ

OpenDJ is an LDAPv3 compliant directory service, which has been developed for the Java platform, providing a high performance, highly available, and secure store for the identities managed by your organization. Its easy installation process, combined with the power of the Java platform makes OpenDJ the simplest, fastest directory to deploy and manage.
https://www.openidentityplatform.org/opendj
Other
368 stars 102 forks source link

encrypt_then_mac TLS extension closes TCP connection #327

Closed WhatANiceChick closed 5 months ago

WhatANiceChick commented 6 months ago

Describe the bug I have an OpenDJ dockered instance listening in LDAPS. I've a LDAPS client on a network device that sends during its Client Hello in the TLS handshake an extension named 'encrypt_then_mac' (see RFC 7366). When receiving the Client Hello, OpenDJ closes the TCP connection by sending a FIN.

image

I got this error in the server.out when adding -Djavax.net.debug=ssl,handshake,trsutmanager in the OPEND_JAVA_ARG :

javax.net.ssl|DEBUG|34|OpenDJ LDAP SDK Grizzly selector thread(4) SelectorRunner|2024-04-17 09:18:23.672 UTC|null:-1|Ignore unknown or unsupported extension (
"encrypt_then_mac (22)": {

}
)
javax.net.ssl|ERROR|34|OpenDJ LDAP SDK Grizzly selector thread(4) SelectorRunner|2024-04-17 09:18:23.673 UTC|null:-1|Fatal (ILLEGAL_PARAMETER): Illegal server name, type=host_name(0), name=**********, value={4853545F3130302E37352E302E38} (
"throwable" : {
  javax.net.ssl.SSLProtocolException: Illegal server name, type=host_name(0), name=*********, value={4853545F3130302E37352E302E38}
        at java.base/sun.security.ssl.ServerNameExtension$CHServerNamesSpec.<init>(Unknown Source)
        at java.base/sun.security.ssl.ServerNameExtension$CHServerNamesStringizer.toString(Unknown Source)
        at java.base/sun.security.ssl.SSLExtension.toString(Unknown Source)
        at java.base/sun.security.ssl.SSLExtensions.toString(Unknown Source)
        at java.base/sun.security.ssl.ClientHello$ClientHelloMessage.toString(Unknown Source)
        at java.base/sun.security.ssl.SSLLogger$SSLSimpleFormatter.formatObject(Unknown Source)
        at java.base/sun.security.ssl.SSLLogger$SSLSimpleFormatter.formatParameters(Unknown Source)
        at java.base/sun.security.ssl.SSLLogger.log(Unknown Source)
        at java.base/sun.security.ssl.SSLLogger.fine(Unknown Source)
        at java.base/sun.security.ssl.ClientHello$ClientHelloConsumer.consume(Unknown Source)
        at java.base/sun.security.ssl.SSLHandshake.consume(Unknown Source)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(Unknown Source)
        at java.base/java.security.AccessController.doPrivileged(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(Unknown Source)
        at org.glassfish.grizzly.ssl.SSLUtils.executeDelegatedTask(SSLUtils.java:227)
        at org.glassfish.grizzly.ssl.SSLBaseFilter.doHandshakeStep(SSLBaseFilter.java:537)
        at org.glassfish.grizzly.ssl.SSLFilter.doHandshakeStep(SSLFilter.java:259)
        at org.glassfish.grizzly.ssl.SSLBaseFilter.doHandshakeSync(SSLBaseFilter.java:442)
        at org.glassfish.grizzly.ssl.SSLBaseFilter.rehandshake(SSLBaseFilter.java:660)
        at org.glassfish.grizzly.ssl.SSLBaseFilter.unwrapAll(SSLBaseFilter.java:374)
        at org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:275)
        at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:246)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:178)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:118)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:96)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
        at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
        at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
        at org.glassfish.grizzly.strategies.SameThreadIOStrategy.executeIoEvent(SameThreadIOStrategy.java:69)
        at org.glassfish.grizzly.strategies.AbstractIOStrategy.executeIoEvent(AbstractIOStrategy.java:66)
        at org.glassfish.grizzly.nio.SelectorRunner.iterateKeyEvents(SelectorRunner.java:381)
        at org.glassfish.grizzly.nio.SelectorRunner.iterateKeys(SelectorRunner.java:353)
        at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:319)
        at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:248)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
        at java.base/java.lang.Thread.run(Unknown Source)
  Caused by: java.lang.IllegalArgumentException: The encoded server name value is invalid
        at java.base/javax.net.ssl.SNIHostName.<init>(Unknown Source)
        ... 40 more
  Caused by: java.lang.IllegalArgumentException: Contains non-LDH ASCII characters
        at java.base/java.net.IDN.toASCIIInternal(Unknown Source)
        at java.base/java.net.IDN.toASCII(Unknown Source)
        ... 41 more}

)

To Reproduce Because this error occurs with a physical firewall, it's hardly reproducable... I think this bug can be reproduce by crafting a TLS handskake using scappy.

Expected behavior I think this TLS extension should be handle.

vharseko commented 5 months ago

The encoded server_name value is invalid (contains non-LDH ASCII characters)