curl / curl

A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl offers a myriad of powerful features
https://curl.se/
Other
35.41k stars 6.37k forks source link

NTLM auth on some IIS servers doesn't work (windows exe) #12511

Closed veodko closed 7 months ago

veodko commented 9 months ago

I did this

curl.exe https://[client's Exchange On-Premise server]/EWS/Exchange.asmx -v --http1.1 -L -k -c cookie.txt -b cookie.txt -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36" --negotiate -u [user_name]

And also many other flags (--ntlm, --negotiate --ntlm, --anyauth)

I expected the following

HTTP 200 response, got HTTP 401 instead.

Opening the link in Chrome (and allegedly, the same request done through Windows shipped version of curl works, I even looked into Chrome's net-internals log and compared the requests, they look the same (also using Negotiate in headers, although I couldn't get to see the values and wasn't allowd to install WireShark on client's machine). Doing the request in PHP's curl 7.70 doesn't work either.

* Host [SECRET] was resolved.
* IPv6: (none)
* IPv4: [SECRET]
*   Trying [SECRET]:443...
* Connected to [SECRET] ([SECRET]) port 443
* ALPN: curl offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 / [blank] / UNDEF
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=[SECRET]
*  start date: Dec 12 00:00:00 2022 GMT
*  expire date: Jan 12 23:59:59 2024 GMT
*  issuer: [SECRET]
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
*   Certificate level 0: Public key type ? (4096/128 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type ? (2048/112 Bits/secBits), signed using sha384WithRSAEncryption
*   Certificate level 2: Public key type ? (4096/128 Bits/secBits), signed using sha384WithRSAEncryption
* using HTTP/1.x
* Server auth using Negotiate with user '[SECRET]'
> GET /EWS/Exchange.asmx HTTP/1.1
> Host: [SECRET]
> Authorization: Negotiate TlR[...]w==
> User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Server: Microsoft-IIS/10.0
< request-id: cc2d07c1-640e-404a-9c29-f5e8785781da
< WWW-Authenticate: Negotiate Tl[...]AAA==
< X-OWA-Version: 15.1.2507.32
< WWW-Authenticate: NTLM
< X-Powered-By: ASP.NET
< X-FEServer: [SECRET]
< Date: Wed, 13 Dec 2023 10:54:04 GMT
< Content-Length: 0
<
* Connection #0 to host [SECRET] left intact
* Issue another request to this URL: 'https://[SECRET]/EWS/Exchange.asmx'
* Found bundle for host: 0x1fbb2497b00 [serially]
* Re-using existing connection with host [SECRET]
* Server auth using Negotiate with user '[SECRET]'
> GET /EWS/Exchange.asmx HTTP/1.1
> Host: [SECRET]
> Authorization: Negotiate TlR[...]9c8=
> User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Server: Microsoft-IIS/10.0
< request-id: 208aedda-58d3-40b4-bd95-757c60dbd8fe
< X-OWA-Version: 15.1.2507.32
< WWW-Authenticate: Negotiate
< WWW-Authenticate: NTLM
< X-Powered-By: ASP.NET
< X-FEServer: [SECRET]
< Date: Wed, 13 Dec 2023 10:54:04 GMT
< Content-Length: 0
<
* Connection #0 to host [SECRET] left intact

I tried decoding the NTLM headers with some python tool (in the order of the requests & responses):

Found NTLMSSP header
Msg Type: 1 (Request)
Domain: '' [] (0b @0)
Workstation: '' [] (0b @0)
OS Ver: '??98????'
Flags: 0x-1df77d49 ["Negotiate Unicode", "Negotiate OEM", "Request Target", "Negotiate Sign", "Negotiate Seal", "Negotiate Lan Manager Key", "Negotiate NTLM", "Negotiate Always Sign", "Negotiate NTLM2 Key", "unknown", "Negotiate 128", "Negotiate Key Exchange", "Negotiate 56"]
Found NTLMSSP header
Msg Type: 2 (Challenge)
Target Name: [SECRET]
Challenge: 0x8e0c0ac74b50668e
Context: '' [] (0b @0)
Target: [block] (136b @66)
    AD domain name (2): [SECRET]
    Server name (1): [SECRET]
    DNS domain name (4): [SECRET]
    FQDN (3): [SECRET]
    Parent DNS domain (5): [SECRET]
    Server Timestamp (7): Ę←z░▓-┌☺
OS Ver: '??98????'
Flags: 0x-1d767dcb ["Negotiate Unicode", "Request Target", "Negotiate Sign", "Negotiate Seal", "Negotiate NTLM", "Negotiate Always Sign", "Target Type Domain", "Negotiate NTLM2 Key", "Negotiate Target Info", "unknown", "Negotiate 128", "Negotiate Key Exchange", "Negotiate 56"]
Found NTLMSSP header
Msg Type: 3 (Response)
LM Resp: u'????????????' [000000000000000000000000000000000000000000000000] (24b @118)
NTLM Resp: 'SECRET' [SECRET] (300b @142)
Target Name: '' [] (0b @88)
User Name: u'[SECRET]' [SECRET] (14b @88)
Host Name: u'[SECRET]' [SECRET] (16b @102)
Session Key: '???"??J??????B??' [SECRET] (16b @442)
OS Ver: '??98????'
Flags: 0x-1d777dcb ["Negotiate Unicode", "Request Target", "Negotiate Sign", "Negotiate Seal", "Negotiate NTLM", "Negotiate Always Sign", "Negotiate NTLM2 Key", "Negotiate

It all worked fine until some security related configuration was changed on the IIS server, which they do not want to change back. That's actually not the first time I encountered this problem, had the same situation earlier but IIS config was restored fortunately. Only feedback I got is that NTLMv1 was disabled and v2 must be used, SSLOffloading disabled and some minor SSL changes (minimum bits). I saw there were some issues previously with NTLMv2 in cURL, but all the issues were closed years ago.

There is no proxy, all requests go directly even in Chrome.

curl/libcurl version

curl 8.5.0, NTLM and SSPI enabled

operating system

Windows 10

veodko commented 9 months ago

I did some more testing on a local IIS setup and could reproduce the problem. On a SSL enabled site once you enable Windows Authentication and then set Extended Protection to Accept or Required, curl stops authenticating (meanwhile it works in chrome). Once you set Extended Protection to Off, curl starts working again. https://learn.microsoft.com/en-us/iis/configuration/system.webserver/security/authentication/windowsauthentication/extendedprotection/

This has been already discussed here and apparently a PR has been merged which should fix this, was there some regression or is it simply some low level Windows thing that can't be fixed by curl? https://github.com/curl/curl/issues/3503

cURL shipped with Windows works and gets HTTP 200 response:

curl 8.4.0 (Windows) libcurl/8.4.0 Schannel WinIDN
Release-Date: 2023-10-11
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS HSTS HTTPS-proxy IDN IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI threadsafe Unicode UnixSockets

compared to binary downloaded from the internet:

curl 8.5.0 (x86_64-w64-mingw32) libcurl/8.5.0 LibreSSL/3.8.2 (Schannel) zlib/1.3 brotli/1.1.0 zstd/1.5.5 WinIDN libssh2/1.11.0 nghttp2/1.58.0 ngtcp2/1.1.0 nghttp3/1.1.0
Release-Date: 2023-12-06
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL SSPI threadsafe UnixSockets zstd

I could just use the binary provided by Microsoft, but the problem is that I'm using the cURL lib in PHP (php_curl.dll) and not directly

vszakats commented 9 months ago

One difference is that Microsoft's curl build uses Schannel by default, while our official builds use LibreSSL (and used quictls/OpenSSL till 8.4.0_8) by default and offer Schannel as a runtime-selectable option.

How does it work for you with this setting?

set CURL_SSL_BACKEND=schannel
bagder commented 9 months ago

I would have guessed on SSPI, but it looks like both versions have that enabled so the SSPI based NTLM implementation should be used for both.

veodko commented 9 months ago

One difference is that Microsoft's curl build uses Schannel by default, while our official builds use LibreSSL (and used quictls/OpenSSL till 8.4.0_8) by default and offer Schannel as a runtime-selectable option.

How does it work for you with this setting?

set CURL_SSL_BACKEND=schannel

It worked! Indeed this was the issue. Is the SSL backend changeable at runtime (to switch to Schannel when using NTLM automatically) or it must be decided at the beginning?

However I'm still troubled with getting it to work in PHP, since it ships cURL without Schannel support at all, I tried setting the env with putenv() and windows env vars but it wouldn't work obviously. Guess I need to re-compile PHP with custom cURL build. I already got to compile cURL with Schannel support, but got some troubles getting it to compile in PHP, no php_curl.dll is generated.

Update: I got it to work, by default PHP is configured to build static libs therefore curl was bundled in the exe, doing configure --with-all-shared --with-curl gave me php_curl.dll and it worked after just replacing that single DLL. I just wonder, is there any downside to using WinSSL/Schannel instead of LibreSSL in the long run? Can something go wrong when doing normal SSL requests?

vszakats commented 9 months ago

In general LibreSSL is more capable than Schannel, and offers these on all Windows versions (e.g. HTTP/3). But depending on your needs Schannel can be just enough.

I'm wondering though why the OpenSSL codepath isn't working for this. It theoretically should, and there may be a bug somewhere.

vszakats commented 9 months ago

It worked! Indeed this was the issue. Is the SSL backend changeable at runtime (to switch to Schannel when using NTLM automatically) or it must be decided at the beginning?

It's the latter.

@veodko: Would you be OK to make a test with the last official version (8.4.0_8) using quictls / OpenSSL (as opposed to LibreSSL)? You can find them here: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48689085/artifacts

This would tell whether this is related to the backend change.

veodko commented 9 months ago

Sure, no problem. Since I'm now working on a test env I can even upload the full log with Authorization headers, if it helps in any way.

Unfortunately it didn't work with curl-8.4.0_8-win32-mingw

curl.exe --version
curl 8.4.0 (i686-w64-mingw32) libcurl/8.4.0 OpenSSL/3.1.4 (Schannel) zlib/1.3 brotli/1.1.0 zstd/1.5.5 WinIDN libssh2/1.11.0 nghttp2/1.58.0 ngtcp2/1.1.0 nghttp3/1.1.0
Release-Date: 2023-10-11
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL SSPI threadsafe UnixSockets zstd
curl.exe https://192.168.1.206 -v -L -k --negotiate -u test:test
*   Trying 192.168.1.206:443...
* Connected to 192.168.1.206 (192.168.1.206) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=DESKTOP-J7QRD3T
*  start date: Dec 15 07:43:04 2023 GMT
*  expire date: Dec 15 00:00:00 2024 GMT
*  issuer: CN=DESKTOP-J7QRD3T
*  SSL certificate verify result: self-signed certificate (18), continuing anyway.
* using HTTP/2
* Server auth using Negotiate with user 'test'
* [HTTP/2] [1] OPENED stream for https://192.168.1.206/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: 192.168.1.206]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAGFKAAAADw==]
* [HTTP/2] [1] [user-agent: curl/8.4.0]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: 192.168.1.206
> Authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAGFKAAAADw==
> User-Agent: curl/8.4.0
> Accept: */*
>
* HTTP/2 stream 1 was not closed cleanly: HTTP_1_1_REQUIRED (err 13)
* Downgrades to HTTP/1.1
* Empty reply from server
* Connection #0 to host 192.168.1.206 left intact
* Issue another request to this URL: 'https://192.168.1.206/'
* Found bundle for host: 0xcafb18 [can multiplex]
* Hostname 192.168.1.206 was found in DNS cache
*   Trying 192.168.1.206:443...
* Connected to 192.168.1.206 (192.168.1.206) port 443
* ALPN: curl offers http/1.1
* SSL reusing session ID
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=DESKTOP-J7QRD3T
*  start date: Dec 15 07:43:04 2023 GMT
*  expire date: Dec 15 00:00:00 2024 GMT
*  issuer: CN=DESKTOP-J7QRD3T
*  SSL certificate verify result: self-signed certificate (18), continuing anyway.
* using HTTP/1.1
* Server auth using Negotiate with user 'test'
> GET / HTTP/1.1
> Host: 192.168.1.206
> Authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAGFKAAAADw==
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: Negotiate TlRMTVNTUAACAAAAHgAeADgAAAA1gori8hQMcMUDme0AAAAAAAAAAJgAmABWAAAACgBhSgAAAA9EAEUAUwBLAFQATwBQAC0ASgA3AFEAUgBEADMAVAACAB4ARABFAFMASwBUAE8AUAAtAEoANwBRAFIARAAzAFQAAQAeAEQARQBTAEsAVABPAFAALQBKADcAUQBSAEQAMwBUAAQAHgBEAEUAUwBLAFQATwBQAC0ASgA3AFEAUgBEADMAVAADAB4ARABFAFMASwBUAE8AUAAtAEoANwBRAFIARAAzAFQABwAIALhM0f8qL9oBAAAAAA==
< Date: Fri, 15 Dec 2023 07:47:48 GMT
< Content-Length: 341
<
* Ignoring the response-body
* Connection #1 to host 192.168.1.206 left intact
* Issue another request to this URL: 'https://192.168.1.206/'
* Found bundle for host: 0xcafb18 [serially]
* Re-using existing connection with host 192.168.1.206
* Server auth using Negotiate with user 'test'
> GET / HTTP/1.1
> Host: 192.168.1.206
> Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAH4AAABAAUABlgAAAAAAAABYAAAACAAIAFgAAAAeAB4AYAAAABAAEADWAQAANYKI4goAYUoAAAAPfke7h/FMJ1AtTk2i8zSAxHQAZQBzAHQARABFAFMASwBUAE8AUAAtAFEAUgAxAEkAVAA3AFQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/giSMKqhdt/nJj73KOmR0QEBAAAAAAAAuEzR/yov2gGXQoCpyH+dHwAAAAACAB4ARABFAFMASwBUAE8AUAAtAEoANwBRAFIARAAzAFQAAQAeAEQARQBTAEsAVABPAFAALQBKADcAUQBSAEQAMwBUAAQAHgBEAEUAUwBLAFQATwBQAC0ASgA3AFEAUgBEADMAVAADAB4ARABFAFMASwBUAE8AUAAtAEoANwBRAFIARAAzAFQABwAIALhM0f8qL9oBBgAEAAIAAAAIADAAMAAAAAAAAAAAAAAAADAAADrxC4xSsujotuXmUTIX3g5atoZpA6OQHqz2yVNF2cEwCgAQAAAAAAAAAAAAAAAAAAAAAAAJACQASABUAFQAUAAvADEAOQAyAC4AMQA2ADgALgAxAC4AMgAwADYAAAAAAAAAAADlXguvkQioMA+4D69iVQOs
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html
< Server: Microsoft-IIS/10.0
< WWW-Authenticate: Negotiate
< WWW-Authenticate: NTLM
< Date: Fri, 15 Dec 2023 07:47:48 GMT
< Content-Length: 1313
<
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1250"/>
<title>401 � Dost�p nieautoryzowany: odmowa dost�pu z powodu nieprawid�owych po�wiadcze�.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;}
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;}
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
 <div class="content-container"><fieldset>
  <h2>401 - Unauthorized: Access is denied due to invalid credentials.</h2>
  <h3>You do not have permission to view this directory or page using the credentials that you supplied.</h3>
 </fieldset></div>
</div>
</body>
</html>
* Connection #1 to host 192.168.1.206 left intact
set CURL_SSL_BACKEND=schannel
curl.exe https://192.168.1.206 -v -L -k --negotiate -u test:test
*   Trying 192.168.1.206:443...
* Connected to 192.168.1.206 (192.168.1.206) port 443
* schannel: disabled automatic use of client certificate
* schannel: using IP address, SNI is not supported by OS.
* ALPN: curl offers h2,http/1.1
* ALPN: server accepted h2
* using HTTP/2
* Server auth using Negotiate with user 'test'
* [HTTP/2] [1] OPENED stream for https://192.168.1.206/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: 192.168.1.206]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAGFKAAAADw==]
* [HTTP/2] [1] [user-agent: curl/8.4.0]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: 192.168.1.206
> Authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAGFKAAAADw==
> User-Agent: curl/8.4.0
> Accept: */*
>
* HTTP/2 stream 1 was not closed cleanly: HTTP_1_1_REQUIRED (err 13)
* Downgrades to HTTP/1.1
* Empty reply from server
* Connection #0 to host 192.168.1.206 left intact
* Issue another request to this URL: 'https://192.168.1.206/'
* Found bundle for host: 0x1495c28 [can multiplex]
* Hostname 192.168.1.206 was found in DNS cache
*   Trying 192.168.1.206:443...
* Connected to 192.168.1.206 (192.168.1.206) port 443
* schannel: using IP address, SNI is not supported by OS.
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.1
* Server auth using Negotiate with user 'test'
> GET / HTTP/1.1
> Host: 192.168.1.206
> Authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAGFKAAAADw==
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: Negotiate TlRMTVNTUAACAAAAHgAeADgAAAA1goriBVrLBZwIGq8AAAAAAAAAAJgAmABWAAAACgBhSgAAAA9EAEUAUwBLAFQATwBQAC0ASgA3AFEAUgBEADMAVAACAB4ARABFAFMASwBUAE8AUAAtAEoANwBRAFIARAAzAFQAAQAeAEQARQBTAEsAVABPAFAALQBKADcAUQBSAEQAMwBUAAQAHgBEAEUAUwBLAFQATwBQAC0ASgA3AFEAUgBEADMAVAADAB4ARABFAFMASwBUAE8AUAAtAEoANwBRAFIARAAzAFQABwAIAMENmYMrL9oBAAAAAA==
< Date: Fri, 15 Dec 2023 07:51:29 GMT
< Content-Length: 341
<
* Ignoring the response-body
* Connection #1 to host 192.168.1.206 left intact
* Issue another request to this URL: 'https://192.168.1.206/'
* Found bundle for host: 0x1495c28 [serially]
* Re-using existing connection with host 192.168.1.206
* Server auth using Negotiate with user 'test'
> GET / HTTP/1.1
> Host: 192.168.1.206
> Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAH4AAABAAUABlgAAAAAAAABYAAAACAAIAFgAAAAeAB4AYAAAABAAEADWAQAANYKI4goAYUoAAAAPV9g+h/fiDPtA9Y88iJAtu3QAZQBzAHQARABFAFMASwBUAE8AUAAtAFEAUgAxAEkAVAA3AFQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFGR3nAK9IcLcTyvAYeI5egEBAAAAAAAAwQ2Zgysv2gHyctSNo5ev1gAAAAACAB4ARABFAFMASwBUAE8AUAAtAEoANwBRAFIARAAzAFQAAQAeAEQARQBTAEsAVABPAFAALQBKADcAUQBSAEQAMwBUAAQAHgBEAEUAUwBLAFQATwBQAC0ASgA3AFEAUgBEADMAVAADAB4ARABFAFMASwBUAE8AUAAtAEoANwBRAFIARAAzAFQABwAIAMENmYMrL9oBBgAEAAIAAAAIADAAMAAAAAAAAAAAAAAAADAAADrxC4xSsujotuXmUTIX3g5atoZpA6OQHqz2yVNF2cEwCgAQAJMl/K66S5NhDW6hrxYyKOEJACQASABUAFQAUAAvADEAOQAyAC4AMQA2ADgALgAxAC4AMgAwADYAAAAAAAAAAABL/mB5aK9ybgwRMrgeemMz
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/html
< Last-Modified: Fri, 15 Dec 2023 07:40:15 GMT
< Accept-Ranges: bytes
< ETag: "4e2e5ef1292fda1:0"
< Server: Microsoft-IIS/10.0
* Negotiate: noauthpersist -> 0, header part: true
< Persistent-Auth: true
< Date: Fri, 15 Dec 2023 07:51:29 GMT
< Content-Length: 696
<
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>IIS Windows</title>
<style type="text/css">
<!--
body {
        color:#000000;
        background-color:#0072C6;
        margin:0;
}

#container {
        margin-left:auto;
        margin-right:auto;
        text-align:center;
        }

a img {
        border:none;
}

-->
</style>
</head>
<body>
<div id="container">
<a href="http://go.microsoft.com/fwlink/?linkid=66138&amp;clcid=0x409"><img src="iisstart.png" alt="IIS" width="960" height="600" /></a>
</div>
</body>
</html>* Connection #1 to host 192.168.1.206 left intact

Same on win32 and win64 build. If there is anything more I could check/log then let me know, perhaps some more verbose OpenSSL debug?

vszakats commented 9 months ago

Thank you @veodko.

If I understand the log correctly, it fails the same way with OpenSSL, so likely applies to any implementation.

I cannot help with NTLM details, but I can cut test releases, if we have a patch to test.

bagder commented 9 months ago

I can't think of many other ways to fix this than to diagnose exactly where in the code paths the working and the non-working versions go separate ways, where they first make different takes where one is wrong and one is right.

NTLM is old, insecure and annoying to work with. In addition to that, it is mostly used by old legacy proprietary Windows servers, which makes it harder to reproduce and test against. As the only known working curl for this setup is using schannel, it also requires curl to run on Windows to debug this case.

Ideally, this live case could be converted into a test case that can be reproduced elsewhere for everyone. That would make it easier to debug and fix.

dfandrich commented 9 months ago

Additionally, Microsoft has just deprecated NTLM.

vszakats commented 9 months ago

NTLM also doesn't build at all with OpenSSL 3 built with its no-deprecated option. The necessary crypto is deprecated. Ref: #12380

bagder commented 7 months ago

Unless we get a recipe to reproduce this issue there is nothing we can do about this. Until then we close this issue.

veodko commented 7 months ago

Well, the recipe to reproduce is already there few messages above and I can't think of any simpler method unfortunately: In IIS, on a SSL enabled site once you enable Windows Authentication and then set Extended Protection to Accept or Required, curl stops authenticating (meanwhile it works in chrome & windows-shipped curl or curl compiled with Schannel). Once you set Extended Protection to Off, curl starts working again. https://learn.microsoft.com/en-us/iis/configuration/system.webserver/security/authentication/windowsauthentication/extendedprotection/

This issue doesn't really bother me anymore though, it works fine since I compiled my own Schannel build and used it to compile PHP later. I don't know if it's worth fixing as people mentioned it is being deprecated, although I doubt it will happen soon as a lot of legacy stuff would probably break so NTLM will still be there for a while.

christophvw commented 4 months ago

I have the same issue.