search

rustls / rustls

A modern TLS library in Rust
Other
5.72k stars 610 forks source link

Reqwest Connection reset by peer, on a Microsoft-IIS/10.0 #1999

Closed GreeFine closed 4 weeks ago

GreeFine commented 4 weeks ago

Checklist

Describe the bug Using reqwest 12.4, I fail to get a server using the rustls feature. If I use native-tls (openssl) it does work.

The server I try to reqwest is the following: https://fdj-prod-scm.hosting.tmmcom.net:8443/ It's a Microsoft-IIS/10.0, and looks like it's supporting: SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 / x25519 / rsaEncryption. Matching Rustls there : https://docs.rs/rustls/latest/rustls/crypto/ring/cipher_suite/static.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384.html

The error I get from the server is a "Connection reset by peer".

reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("fdj-prod-scm.hosting.tmmcom.net")), port: Some(8443), path: "/", query: None, fragment: None }, source: hyper_util::client::legacy::Error(SendRequest, hyper::Error(Io, Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" })) }

What I don't understand is when looking at it with wireshark it looks like the TLS negociation went well, it's rigth after that the server close the connection unexpectedly:

image

To Reproduce Steps to reproduce the behavior:

[dependencies]
reqwest = { version = "0.12", default-features = false, features = [
  "blocking",
  "rustls-tls",
  # "native-tls",
] }
fn main() {
    let client = reqwest::blocking::Client::builder()
        .http1_only()
        .build()
        .unwrap();

    dbg!(client
        .get("https://fdj-prod-scm.hosting.tmmcom.net:8443/")
        .send()
        .unwrap());
}

If you run this code with the rustls-tls feature for reqwest, it doesn´t work, but replacing the feature with native-tls works.

Applicable Version(s) Linux

[[package]]
name = "rustls"
version = "0.22.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf4ef73721ac7bcd79b2b315da7779d8fc09718c6b3d2d1b2d94850eb8c18432"
dependencies = [
 "log",
 "ring",
 "rustls-pki-types",
 "rustls-webpki",
 "subtle",
 "zeroize",
]

Expected behavior I expect reqwest to be able to receive the response from the server when using rustls instead of openssl

Additional context

RUST_LOG=trace cargo run ```rs Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.08s Running `target/debug/test-simple` TRACE reqwest::blocking::client > (ThreadId(2)) start runtime::block_on TRACE reqwest::blocking::wait > (ThreadId(1)) park without timeout TRACE reqwest::blocking::wait > wait at most 30s TRACE reqwest::blocking::wait > (ThreadId(1)) park timeout 29.999998633s DEBUG reqwest::connect > starting new connection: https://fdj-prod-scm.hosting.tmmcom.net:8443/ DEBUG rustls::client::hs > No cached session for DnsName("fdj-prod-scm.hosting.tmmcom.net") DEBUG rustls::client::hs > Not resuming any session TRACE rustls::client::hs > Sending ClientHello Message { version: TLSv1_0, payload: Handshake { parsed: HandshakeMessagePayload { typ: ClientHello, payload: ClientHello( ClientHelloPayload { client_version: TLSv1_2, random: 8f543fd02f7c80ad342475281453f9882a13154c7d701f7c5ced8affb38269fb, session_id: e566bd615b8f3c7ecc28b5a2e9b739090407aeecd8d380915a941540d30f0372, cipher_suites: [ TLS13_AES_256_GCM_SHA384, TLS13_AES_128_GCM_SHA256, TLS13_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, ], compression_methods: [ Null, ], extensions: [ SupportedVersions( [ TLSv1_3, TLSv1_2, ], ), EcPointFormats( [ Uncompressed, ], ), NamedGroups( [ X25519, secp256r1, secp384r1, ], ), SignatureAlgorithms( [ ECDSA_NISTP384_SHA384, ECDSA_NISTP256_SHA256, ED25519, RSA_PSS_SHA512, RSA_PSS_SHA384, RSA_PSS_SHA256, RSA_PKCS1_SHA512, RSA_PKCS1_SHA384, RSA_PKCS1_SHA256, ], ), ExtendedMasterSecretRequest, CertificateStatusRequest( Ocsp( OcspCertificateStatusRequest { responder_ids: [], extensions: , }, ), ), ServerName( [ ServerName { typ: HostName, payload: HostName( DnsName( "fdj-prod-scm.hosting.tmmcom.net", ), ), }, ], ), KeyShare( [ KeyShareEntry { group: X25519, payload: bedc40720466df01eeb406080e17008959fe0e55fce933975c83ca3a5edcbc7d, }, ], ), PresharedKeyModes( [ PSK_DHE_KE, ], ), Protocols( [ ProtocolName( 687474702f312e31, ), ], ), SessionTicket( Request, ), ], }, ), }, encoded: 0100010803038f543fd02f7c80ad342475281453f9882a13154c7d701f7c5ced8affb38269fb20e566bd615b8f3c7ecc28b5a2e9b739090407aeecd8d380915a941540d30f03720014130213011303c02cc02bcca9c030c02fcca800ff010000ab002b00050403040303000b00020100000a00080006001d00170018000d001400120503040308070806080508040601050104010017000000050005010000000000000024002200001f66646a2d70726f642d73636d2e686f7374696e672e746d6d636f6d2e6e6574003300260024001d0020bedc40720466df01eeb406080e17008959fe0e55fce933975c83ca3a5edcbc7d002d000201010010000b000908687474702f312e3100230000, }, } TRACE rustls::client::hs > We got ServerHello ServerHelloPayload { legacy_version: TLSv1_2, random: 6668562fc20eccd1cd83e56851f49a37061bed7872e45177c0b4f558d0f06076, session_id: 233000006b1d24238d7df4996055bb5480a3d4467f88476b4469982d6c839b01, cipher_suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, compression_method: Null, extensions: [ Protocols( [ ProtocolName( 687474702f312e31, ), ], ), ExtendedMasterSecretAck, RenegotiationInfo( , ), ServerNameAck, ], } DEBUG rustls::client::hs > ALPN protocol is Some(b"http/1.1") DEBUG rustls::client::hs > Using ciphersuite TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 DEBUG rustls::client::tls12 > ECDHE curve is EcParameters { curve_type: NamedCurve, named_group: X25519 } TRACE rustls::client::tls12 > Server cert is CertificateChain([CertificateDer(0x3082065930820541a003020102020c71715fc2e6b93139d3bc7830300d06092a864886f70d01010b0500304c310b300906035504061302424531193017060355040a1310476c6f62616c5369676e206e762d73613122302006035504031319416c70686153534c204341202d20534841323536202d204734301e170d3233303631333135313633365a170d3234303731343135313633355a301f311d301b06035504030c142a2e686f7374696e672e746d6d636f6d2e6e657430820122300d06092a864886f70d01010105000382010f003082010a0282010100bbc64b5fd2f21acb2d9e543e1d22ea1feb7ad64a95684684097310cacd6a6994b2ff324a72b3db2a258b1942ec5b5fb1f026de503870774d86c63b5f6453fec84fb7c391637e84f0e01e245357c754419d7d4ed36c943be6c16612b732ab870a08889c72e6445732c81d493a1cc5e229a2c0ecfb6bd0feeb4578b064a4eb7185d61e46f97f0e73f7b9bbc3d984e9214d06cedb042ef5f854ee0b270e2a56180c39e0fd53b97fd3662a4250bb244e93a4fb187a76ecc1ee06458b7a8fa467956b20a94807a877882593f94213383e22261c3d38a61c0665f7cc5ad98c8145dc3a06584719a32d05c80a1b96f25a73c7adeace421e0bf711ea0123b30332ee3f3b0203010001a382036630820362300e0603551d0f0101ff0404030205a030819306082b06010505070101048186308183304606082b06010505073002863a687474703a2f2f7365637572652e676c6f62616c7369676e2e636f6d2f6361636572742f616c70686173736c636173686132353667342e637274303906082b06010505073001862d687474703a2f2f6f6373702e676c6f62616c7369676e2e636f6d2f616c70686173736c6361736861323536673430570603551d200450304e3008060667810c0102013042060a2b06010401a0320a01033034303206082b06010505070201162668747470733a2f2f7777772e676c6f62616c7369676e2e636f6d2f7265706f7369746f72792f30090603551d130402300030410603551d1f043a30383036a034a0328630687474703a2f2f63726c2e676c6f62616c7369676e2e636f6d2f616c70686173736c636173686132353667342e63726c30330603551d11042c302a82142a2e686f7374696e672e746d6d636f6d2e6e65748212686f7374696e672e746d6d636f6d2e6e6574301d0603551d250416301406082b0601050507030106082b06010505070302301f0603551d230418301680144fcbaca8c2efabdd836f6bbfce983d5c58257615301d0603551d0e0416041423e8181e637f5be651e3a67f8dd6a87912985afe3082017d060a2b06010401d6790204020482016d04820169016700760048b0e36bdaa647340fe56a02fa9d30eb1c5201cb56dd2c81d9bbbfab39d8847300000188b5549b610000040300473045022100deb4413d8b560eb85ee0ed52283519fdfdab1a5a8d75e9b6ae4de1853c9e824c022009a97645f1c9e3b5abc464dd3ece134f8ab93ef7625270908d8038d68328b8cf007600eecdd064d5db1acec55cb79db4cd13a23287467cbcecdec351485946711fb59b00000188b5549b500000040300473045022100f024e698c7291fae5a6512b906bff40a4330c5537c2d106e1faf1b32e60e842602202a812ec9abff92210561329442dffadce74c40eebe146631abb696b92c93745200750076ff883f0ab6fb9551c261ccf587ba34b4a4cdbb29dc68420a9fe6674c5a3a7400000188b5549b920000040300463044022061e295f92f7e02c4a04cccce399c514283d450a93a545bb711639d80e645cd68022024bf92dde0cfb1aa9db863a5a50d876dac02bbb43c37849fe716bf482347597d300d06092a864886f70d01010b05000382010100a8150d4a1f9992fe29ef476a7f78eae3de96843d8a27595fcb698d92fdf33e1aff4afe39608062e4b86fcb7d416647bc3b71701aba8519cf1edb24551f540c94c9f197888d6455b00db069e7abf6293c795fd6e645fddae666ad15846dc365fc234dbcb54aff1b08ef6f3dc2f8660f6fe867e138b3eb26e7ecf42a1146392c7cc73344e15f83bcae6534cddc06a6700990cecaadc8d6953a2d689323f82d6c38639bf251b35a552d8440e841da0028b15901335d06fd75f13630a1210a480a12e7ec89a59105511c27e8a8fdb78da02cf41663cce7f62f6ae0ad385c4c711aeab1489910a32e65812f4708d57c9ebd2b07df1d1f75c578c8d71cdc264228424d), CertificateDer(0x3082048a30820372a00302010202107d4d42a92b431d7e6453e7c19a8d5877300d06092a864886f70d01010b05003057310b300906035504061302424531193017060355040a1310476c6f62616c5369676e206e762d73613110300e060355040b1307526f6f74204341311b301906035504031312476c6f62616c5369676e20526f6f74204341301e170d3232313031323033343934335a170d3237313031323030303030305a304c310b300906035504061302424531193017060355040a1310476c6f62616c5369676e206e762d73613122302006035504031319416c70686153534c204341202d20534841323536202d20473430820122300d06092a864886f70d01010105000382010f003082010a0282010100ad2429956615883f33870378cfd50c24b83153f3ff83226c99952b7ce54a59c2aec6d12a9dfa7f202e51c8672a5091a7795644fb38b53e308efc942ecb570c69535f44c656962faec0372586f171f1dc0245428661b836ef51e373450c90b3a5d2e7037ab83945d017f502d094416ac618b198c320b5c53af382b14aa444ac21732a9255064ec87c8bb0ca66145455f82b3cb25491b6cb52b2d8e36f8a4428b07d2bc19680b93e00d89e3de8319d5a4dedd67e4de5d48e03dd129a2783d4d6a1d784724e81ed9b8c620697a32c68137e041dacafa127c57d319cc21b7b0da821f385a0baace3bbe1fc61f824dd2aaa5d960477c33d50e6ddbf8643163a37f2d70203010001a382015b30820157300e0603551d0f0101ff040403020186301d0603551d250416301406082b0601050507030106082b0601050507030230120603551d130101ff040830060101ff020100301d0603551d0e041604144fcbaca8c2efabdd836f6bbfce983d5c58257615301f0603551d23041830168014607b661a450d97ca89502f7d04cd34a8fffcfd4b307a06082b06010505070101046e306c302d06082b060105050730018621687474703a2f2f6f6373702e676c6f62616c7369676e2e636f6d2f726f6f747231303b06082b06010505073002862f687474703a2f2f7365637572652e676c6f62616c7369676e2e636f6d2f6361636572742f726f6f742d72312e63727430330603551d1f042c302a3028a026a0248622687474703a2f2f63726c2e676c6f62616c7369676e2e636f6d2f726f6f742e63726c30210603551d20041a30183008060667810c010201300c060a2b06010401a0320a0103300d06092a864886f70d01010b050003820101001a25f673648840a95907a743ba153f5161bd15ff2d64ddcd7a5d326a7f4842e710986839efb7eba13476df2d58683e7b301c0cf78660f9a9f379c054b783a638bb36abbc95d07cf86fc1e94f4607c8b60c3200a92b0512f70c6d66f9819dbf0e644d7227c68bd14a02e16edb0c9fb78b380c7c332f6089db38cc95438cdd1684d5cc6e3acf8e9ba3020fd1bbbe7900b52882fce39f1cef74d9fe322366b8f0afa029a01fde52121578dddf6a70436d4ba4cdee7881b275a27ed7fcfc9eff82ed2513e5b1e8cfb718536ecb52f8759f65923670bafd0c054a83fa80d29ae0f38efe83b5df18e1acb44727fd3870a31b4402ed2564243da709f12255841d91ec12)]) DEBUG rustls::client::tls12 > Server DNS name is DnsName("fdj-prod-scm.hosting.tmmcom.net") WARN rustls::common_state > Sending warning alert NoRenegotiation DEBUG rustls::common_state > Sending warning alert CloseNotify thread 'main' panicked at src/main.rs:11:10: called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("fdj-prod-scm.hosting.tmmcom.net")), port: Some(8443), path: "/", query: None, fragment: None }, source: hyper_util::client::legacy::Error(SendRequest, hyper::Error(Io, Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" })) } note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace TRACE reqwest::blocking::client > closing runtime thread (ThreadId(2)) TRACE reqwest::blocking::client > signaled close for runtime thread (ThreadId(2)) TRACE reqwest::blocking::client > (ThreadId(2)) Receiver is shutdown TRACE reqwest::blocking::client > (ThreadId(2)) end runtime::block_on TRACE reqwest::blocking::client > (ThreadId(2)) finished TRACE reqwest::blocking::client > closed runtime thread (ThreadId(2)) ```
ctz commented 4 weeks ago 
 WARN  rustls::common_state      > Sending warning alert NoRenegotiation

We do this in accordance with RFC5246 to reject a server's attempts to start a renegotiation -- I guess IIS treats this as fatal?

We don't implement renegotiation for reasons described in the manual.

cpu commented 4 weeks ago

Thank you for the very detailed report!

I took a pcap using SSLKEYLOGFILE so the post-handshake messages could be decrypted and it matches ctz's theory. The server sends a Hello Request message, Rustls replies with the NoRenegotiation warning, and the remote peer closes abruptly w/ a TCP reset:

cap

GreeFine commented 4 weeks ago

:face_holding_back_tears: really happy to have an explanation, thank ctz and cpu for taking the time.

I have a workaround, but I will try to see if I can skip the warning and make it work. If it does, would it be a good idea to open a PR, or should we just say that we don't want to support this kind of behavior ?

ctz commented 4 weeks ago

I don't think we'd accept a PR that either disables the NoRenegotiation warning alert behaviour, or implements TLS1.2 renegotiation. My advice would be to ensure the server can negotiate TLS1.3.

cpu commented 4 weeks ago

I don't think we'd accept a PR that either disables the NoRenegotiation warning alert behaviour

I think even if this perspective changed (I'm in agreement with you FWIW) I don't think it would get you to a full solution for communicating with this server. It sounds like IIS may use TLS 1.2 renegotiation to signal that a client certificate is required conditionally based on the initial handshake's HTTP request (ref https://security.stackexchange.com/a/24569). If that's true I suspect the only fix would be to implement TLS 1.2 renegotiation so that your Rustls client sends a second client hello with the mTLS client certificate details that IIS expects. A quick experiment disabling the alert shows the connection hanging without an HTTP response after the hello request arrives and is ignored.

Given the risky security properties of renegotiation support and the legacy status of TLS 1.2 it feels like a very tough sell to invest resources in a fix. Sorry for the bad news :'( Using native-tls when you need to communicate with this server profile might be the best option.

lvkv commented 1 week ago

@GreeFine I ran into a very similar issue to yours—possibly the same one. @ctz and @cpu are right that IIS uses renegotiation as a client authentication mechanism; I assume this is to handle selectively requiring client authentication post-handshake depending on the requested resource. Either way, you can avoid this by reconfiguring the server to always request client certificates.

  1. Note the settings for your SSL certificate bindings with netsh http show ssl. Take note of IP:port, Certificate Hash, Application ID and Negotiate Client Certificate, which I'll assume is Disabled. Also take note of DS Mapper Usage if it's Enabled—enabling client cert negotiation may disable this as a side effect.

    1. In an elevated shell, enter a netsh session and enable client cert negotiation (changing values as needed): 
      
PS C:\Windows\system32> netsh
netsh>http update sslcert ipport=$ipport certhash=$certhash appid=$appid clientcertnegotiation=enable

    SSL Certificate successfully updated

    
If you get `The parameter is incorrect` then your shell is probably not elevated or your invocation isn't correct. You may or may not have to restart IIS.
  2. (Optional) Re-enable DS Mapper if it was enabled previously using the same http update method above. Do a bunch of nonsense to make the other nonsense work. Ensure that your turbo encabulator's base plate of pre-famulated amulite (surmounted by malleable logarithmic casing) still ensures that the two spurving balls are directly in line with the panametric fan.

Looking at a decrypted packet capture confirms the change—the server now sends a proper certificate request as part of its server hello, instead of e.g. after the client GET or POST-ing first, and I can finally dump OpenSSL and use Rustls instead.