search

clj-commons / aleph

Asynchronous streaming communication for Clojure - web server, web client, and raw TCP/UDP
http://aleph.io
MIT License
2.54k stars 241 forks source link

Increase TCP SSL Buffer #637

Closed MichaelALacefield closed 1 year ago

MichaelALacefield commented 2 years ago

My error is aleph.tcp - error in TCP client io.netty.handler.codec.DecoderException: javax.net.ssl.SSLProtocolException: Input record too big: max = 16709 len = 18405

I am struggling to increase the allowed size I've tried using {:bootstrap-transform (fn [bs] (.option bs ChannelOption/SO_SNDBUF (int 32767)) (.option bs ChannelOption/SO_RCVBUF (int 32767)) bs)}

but no luck. Any advice would be appreciated. Thank you.

arnaudgeiser commented 2 years ago

Hello Michael!

I don't think it's something that can be configured inside Netty (I may be wrong there).

Can I ask you about your Java version? It seems there was a bug on the JDK at some point. [1] What is your operating system? I found some references about this issue on Microsoft about the size of the trusted CA. [2] Also, it seems a TLS record cannot be more than 16 kilobytes, are you sure about what you receive?

Sorry for the non-answer...

[1] : https://bugs.openjdk.org/browse/JDK-8227651 [2] : https://learn.microsoft.com/en-us/troubleshoot/windows-server/windows-security/ssltls-communication-problems-after-install-kb931125 [3] : https://hpbn.co/transport-layer-security-tls/

KingMob commented 2 years ago

We're also pretty active on the #aleph channel on the Clojurians Slack.

KingMob commented 2 years ago

@MichaelALacefield One thing to note, if it is in fact bug JDK-8227651, that affects Java versions from 8-19, and was only fixed a few weeks ago. So it's not about upgrading the major version, you'll have to get the latest point releaese.

DerGuteMoritz commented 2 years ago

I am struggling to increase the allowed size I've tried using {:bootstrap-transform (fn [bs] (.option bs ChannelOption/SO_SNDBUF (int 32767)) (.option bs ChannelOption/SO_RCVBUF (int 32767)) bs)}

Note that the options you're setting here affect the underlying TCP connection while the error you're seeing is on the TLS level above it (cf. @arnaudgeiser comment above). I would also recommend upgrading your JDK to see if that fixes it.

That said, there's another (old) issue which is relevant to this: https://bugs.openjdk.org/browse/JDK-6388456 - it reiterates that 16 KiB is the max allowed message size as per the spec. AFAIUI they introduced a way to tweak this limit to be able to communicate with non-conforming servers but unfortunately, the issue doesn't mention how exactly. Might be worth checking the SSLEngine docs (unless the JDK upgrade already does the trick, that is).

MichaelALacefield commented 2 years ago

I don't think it's something that can be configured inside Netty (I may be wrong there).

Can I ask you about your Java version? It seems there was a bug on the JDK at some point. [1] What is your operating system? I found some references about this issue on Microsoft about the size of the trusted CA. [2] Also, it seems a TLS record cannot be more than 16 kilobytes, are you sure about what you receive?

Thanks for the reply I'm running on centos 7, openJDK version 11.0.16, and ya we used to have a connection to the device with our own socket client and we were able to receive the full file the files we could receive can be up to 64k in length.

Note that the options you're setting here affect the underlying TCP connection while the error you're seeing is on the TLS level above it (cf. @arnaudgeiser comment above). I would also recommend upgrading your JDK to see if that fixes it.

Thanks for that note and ya if I take the file and count the size in notepad its 48k characters long not sure where it gets the len of 18405.

I dont know if its a Java issue because when we used java.net socket to create our own client it would work out. but I will give it a shot.

We're also pretty active on the #aleph channel on the Clojurians Slack.

Awesome I'll have to join the channel.

Thanks everyone

DerGuteMoritz commented 2 years ago

Thanks for the reply I'm running on centos 7, openJDK version 11.0.16

Then there's hope since the fix for JDK-8227651 only went in with 11.0.18.

Thanks for that note and ya if I take the file and count the size in notepad its 48k characters long not sure where it gets the len of 18405.

That's just the length of the offending TLS record, i.e. one chunk of your transfer.

I dont know if its a Java issue because when we used java.net socket to create our own client it would work out. but I will give it a shot.

As mentioned in this comment the issue only occurs with TLS 1.3 due to how the handshake works very differently compared to pervious versions. It might be that your java.net.Socket based implementation didn't use TLS 1.3 for some reason or perhaps the timing was slightly different so that you didn't encounter the bug there (as you can see in the linked comment, the underlying cause is a race condition).

Let us know if upgrading your JDK resolved the issue!

DerGuteMoritz commented 2 years ago

Ah damn, just noticed that 11.0.18 hasn't been released, yet. An interim workaround then should be to use TLS 1.2 if the server allows it.

arnaudgeiser commented 2 years ago

Any news from the front @MichaelALacefield?

MichaelALacefield commented 1 year ago

Sorry for no update tried spinning up a new environment with newer java and still no luck, we are currently looking at upgrading our linux distro from centos 7 which will make it easier to pick a newer java and then I'll be trying to fix this again. I'll hit you guys up on stack if I need it Thank you. Also tried TLS 1.2 but no luck the server we are hitting has a different request we can use to specify how many bytes we want to receive from the request so I can use that as a work around