steffengy / schannel-rs

Schannel API-bindings for rust (provides an interface for native SSL/TLS using windows APIs)
MIT License
46 stars 50 forks source link

TlsStream server is busy-waiting if TLS handshake is delayed by client #88

Open jiuka opened 1 year ago

jiuka commented 1 year ago

I tried to build a async wrapper around schannel::tls_stream::Builder to build a TLS server which can make use of certificates from the Windows CertStore. The wrapper works when used as expected. However while testing I encountered a problem that the thread/task handling the MidHandshakeTlsStream would block if a TCP connection is opened but no handshake started.

While debugging I found out that the the call to accept would end up in a busy-loop resulting the process using as much CPU as possible if a TCP connection is opened but no handshake takes place. (For example opening a telnet connection to the listening socket.) This even happens if I ditch my async wrapper and use the schannel in a synchronous fashion.

MidHandshakeTlsStream::handshake calls TlsStream::initialize which in turn calls TlsStream::step_initialize in a enless loop. As there is no data ready for a handshake yet the call to Identity::AcceptSecurityContext returns a status of Foundation::SEC_E_INCOMPLETE_MESSAGE which ends up with needs_read to being set to 0. With needs_read as 0 there is of cours no need to return a WouldBlock error so Identity::AcceptSecurityContext is called again right away in a endless loop.

If I changed the Foundation::SEC_E_INCOMPLETE_MESSAGE arm to always set needs_read to at least one, this busy-loop was broken and the process would not hog the cpu but wait for data being received in the sync case or yield to other tasks in the async case.

While I think this is a bug chances are that I am just to inexperienced with rust and schannel and just made a fool out of myself.