Closed argv-minus-one closed 1 year ago
Yeah, that problem sounds like it's coming from the schannel crate in particular.
As of the Windows 2022-11 cumulative update, this bug seems to have disappeared. The test program now works correctly on both of the Windows 10 machines I've tried running it on. Looks like Microsoft fixed it, so I'll go ahead and close this.
I want to use
native-tls
in a client-side program to talk to an OpenSSL server. For the most part, this works fine, except when:schannel
vianative-tls
on Windows, or if it uses OpenSSL 1.1.1.)Under these circumstances, the
native-tls
-based client slowly allocates up to 4GB of memory, then hangs.Here's a GitHub repository with a simple test program that demonstrates the problem. To use it:
openssl
command is available and is OpenSSL 3 (checkopenssl version
). On Windows, I've tried with the FireDaemon binaries, which demonstrate the problem. (The test program works correctly with FireDaemon's OpenSSL 1.1.1 build; it only fails with OpenSSL 3.)cd
into it.cargo run --bin openssl_server -- 127.0.0.1:4433
- this generates some certificates and then runsopenssl s_server
.ACCEPT
when it's ready.cargo run --bin native_tls_client -- 127.0.0.1:4433
- this starts up anative-tls
client that tries to connect to the server and send it an HTTP request. If it succeeds, it writes the response to stdout.If you do this on Linux, it works fine. If you do this on Windows and the server uses OpenSSL 1.1.1, it works fine. If you do this on Windows and the server uses OpenSSL 3, the client hangs.
I noticed that the
curl
build that comes with Windows, which I think also uses the SChannel API, works fine even if the server is running OpenSSL 3.I ran this in a debugger and found that the 4GB memory allocation is being done by the
schannel
crate. Somehow, whenschannel::tls_stream::TlsStream::step_initialize
callsInitializeSecurityContextW
, the latter function setsinbufs[1].cbBuffer
to just underu32::MAX
(in my debugger right now, it's 4294966277).TlsStream::read_in
then allocates and initializes that much memory, which on my machine takes a minute or two, and then hangs trying to read that many bytes from the socket.So, maybe the issue is with the
schannel
crate rather thannative-tls
, but I'm not sure, so I thought I'd post an issue here first and see what you think.