NTLM credentials not sent by client when there are multiple WWW-Authenticate headers

[aplex]

We have a strange case after migrating from RC1 to RC2. We use HttpClient with default windows credentials authentication. The code works fine with one server, but does not work with another server. In the second case client just receives 401 error and does not start NTLM handshake.

The only difference that I spotted was that second server sends two WWW-Authenticate headers (one with basic, another with HTML).

When I intercepted the response with Fiddler, and removed "WWW-Authenticate" header with basic authorization, everything worked fine, client started NTLM handshake, and finally authorized.

var handler = new HttpClientHandler();
handler.UseDefaultCredentials = true;            
var client = new HttpClient(handler);
HttpResponseMessage response = await client.GetAsync(url);

Request/response from second server:

GET https://my-server:8081/tfs/DefaultCollection/_apis/projects?api-version=1.0 HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Host: my-server:8081

HTTP/1.1 401 Unauthorized
Content-Type: text/html
Server: Microsoft-IIS/8.5
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 3600
Access-Control-Allow-Methods: OPTIONS,GET,POST,PATCH,PUT,DELETE
Access-Control-Expose-Headers: ActivityId,X-TFS-Session,X-MS-ContinuationToken
Access-Control-Allow-Headers: authorization
X-FRAME-OPTIONS: SAMEORIGIN
Set-Cookie: Tfs-SessionId=XXX; path=/; secure
Set-Cookie: Tfs-SessionActive=2016-06-07 20:44:23Z; path=/; secure
WWW-Authenticate: Basic realm="my-server"
WWW-Authenticate: NTLM
P3P: CP="CAO DSP COR ADMa DEV CONo TELo CUR PSA PSD TAI IVDo OUR SAMi BUS DEM NAV STA UNI COM INT PHY ONL FIN PUR LOC CNT"
X-Content-Type-Options: nosniff
Date: Tue, 07 Jun 2016 20:44:22 GMT
Content-Length: 1293

OS: windows 10 x64, app running on coreclr RC2

[davidsh]

cc: @stephentoub

Could be related to dotnet/runtime#17508.

[davidsh]

You code is not turning on default credentials. Is that a typo?

handler.UseDefaultCredentials = false;  

[aplex]

yep, that was a typo. In reality the code is a little bigger - with some user agent headers, and other stuff.

[aplex]

I see this is scheduled for 1.1.0. Is there any workaround available? Maybe there is some place where I could intercept the request and remove the second WWW-Authenticate header before framework handles 401 response?

[davidsh]

@aplex

I tried to repro your scenario but I couldn't repro. I set up an IIS server that uses both ntlm and basic. I was able to connect using default credentials.

One thing I noticed, though, is that my IIS server sends out the 'WWW-Authenticate' headers using NTLM first and then basic:

WWW-Authenticate: NTLM WWW-Authenticate: Basic realm="testserver"

It shouldn't matter, though, since NTLM will be preferred regardless of what order the server sends them out.

Can you retry your server repro and see if it is different if you have NTLM first and then Basic?

[aplex]

Ok, so I tried to change the order of WWW-Authenticate headers using Fiddler, so that NTLM was first, and client continued with NTLM handshake. I guess it just picks the first header then?

If you need some way to check it, you can use Fiddler with before-request breakpoints that allows to change the 401 response from server before it gets to the client.

[davidsh]

Thx for confirming this. Also, you mentioned that this worked in RC1 but then broke in RC2. Is that correct?

[davidsh]

One other thing to try....are you going thru a proxy? Try turning off proxies completely by setting .UseProxy=false.

I have reproduced a problem with getting headers in the "Basic", "NTLM" order and using a proxy (such as Fiddler). It appears that NTLM handshakes don't go thru.

[davidsh]

cc: @stephentoub @CIPop

This looks like a native WinHTTP bug. I opened up a bug with the WinHTTP team. Ref: 7924548

[aplex]

Disregard that thing about RC1 - I used full .Net framework there, so that's probably the reason it worked.

I tried with handler.UseProxy = false and result was the same - 401 error

[gerab]

wow, i spent two days with debugger and wireshark to find the same bug. The workaround for me was to offer server all possible authorization schemes:

_credentialCache = new CredentialCache
            {
                {new Uri(ResourceUrls.BaseUrl), "NTLM", _credential},
                {new Uri(ResourceUrls.BaseUrl), "NEGOTIATE", _credential}
            };

or to remove all providers from iis except ntlm. That is not prefered as may cause some issues later .

[karelz]

@davidsh is there something we could do to compensate / detect prior/post the bad behavior in the OS? If we can bring the 2 days tracking time down, that would be amazing ... @gerab any idea what would have helped you? (beside having the OS bug fixes)

[davidsh]

@gerab Are you going thru a proxy? The original bug report could only be repro'd if going thru a proxy.

[gerab]

@davidsh no proxy between me and server

[davidsh]

I was able to repro the problem now without using a proxy.

For example, TFS servers will present WWW-Authenticate headers listing Basic first and NTLM, Negotiate last:

WWW-Authenticate: Bearer WWW-Authenticate: Basic realm="http://exampleTFSServer:8080/" WWW-Authenticate: Negotiate WWW-Authenticate: NTLM

I am following up with WinHTTP team. They previously closed the bug as not-repro.

[davidsh]

@gerab Also, is there any kind of redirection involved on the server endpoint you are using. The default behavior in the handlers is to provide for automatic redirection. But I think the root cause problem is not the order of authenticate headers but rather if redirection is involved. For example, in the TFS server I mentioned above, there is redirection to a subfolder. If I do an HTTP request directly to that subfolder, I don't see a problem.

[davidsh]

I have root caused what I believe to be the real problem here. Our WinHttpHandler currently clears out all credentials (including specifying the default credential) if there is redirection.

See: https://github.com/dotnet/corefx/blob/master/src/System.Net.Http.WinHttpHandler/src/System/Net/Http/WinHttpRequestCallback.cs#L210

But that is too strict. We should only be clearly this out if we change host:port. If we are just going down into a subfolder, we need to keep the authentication as-is. That would then match .NET Framework as well.

I also think we need more tests in CI around this topic.

cc: @stephentoub

[davidsh]

Moving to 2.0 milestone.

[davidsh]

Also, I no longer think this is an external WinHTTP issue.

[gerab]

@davidsh that was a request from HttpClient to web api hosted by kestrel behind iis. Does it matter? iis acts like proxy

[davidsh]

@davidsh that was a request from HttpClient to web api hosted by kestrel behind iis. Does it matter? iis acts like proxy

When I asked about a "proxy", I wasn't talking about in the "architecture" sense. Yes, that architecture could be thought of as a proxy.

But I was talking about the literal sense of whether HttpClient is sending a request on the wire and there is a proxy between it and the uri it is sending the request to.

[gerab]

@davidsh i'm sorry, for that misunderstanding David. I am absolutely sure, that there is no proxy between me and server

[davidsh]

@gerab Can you comment on whether there are redirections in your repro? I believe the root cause in the repros is due to redirection and our bug (as I noted above) w.r.t. how we handle credentials after redirection

[stephentoub]

But that is too strict. We should only be clearly this out if we change host:port. If we are just going down into a subfolder, we need to keep the authentication as-is.

I believe CurlHandler copied this logic from WinHttpHandler; if there's an issue in WinHttpHandler to be fixed, we likely have a similar one in CurlHandler.

I also think we need more tests in CI around this topic.

The more tests the better, and that'll help highlight if we actually have a problem on Unix as well.

[danmoseley]

Triage believe this won't be commonly encountered (based on data we have). Moving to Future. @DavidGoll can you please note any workaround here if there is one.

[DavidGoll]

Workaround is as specified by @gerab above.

[aplex]

I can't agree that it is not commonly encountered. The original problem was with connection to Microsoft TFS from a .Net core application. I am not completely sure, but I think TFS was also in a default configuration.

Changing authentication settings in IIS/TFS is not a good option, as it may break other applications that depend on them.

Workaround with CredentialCache is not very clear either. What url should be put there - for specific request or top-level url for the server? Should we also put "Basic" scheme there, in case NTLM is disabled? Should we include realm in "basic" scheme? Will this CredentialCache work the same way on Mac and Linux?

[karelz]

@aplex the API exists since .NET Core 1.0, yet we had only 2 reports so far -- that's why we said "not commonly encountered". If you have evidence there are more people hitting it without us knowing (e.g. StackOverflow discussions), please let us know. If the impact on all .NET Core customers is larger than we thought, we can always reconsider. Thanks!

[sbraswell]

@karelz I've run into this issue. We have ported a Windows 8.1 store app to UWP and the same code that worked in Windows 8.1 doesn't work in UWP. I've used the same method described by @aplex to modify the response using Fiddler. I've found that if I reorder the WWW-Authenticate headers so NTLM is first, the HttpClient will continue with the authentication conversation. If anything other than NTLM is listed first then HttpClient.GetStringAsync(...) throws an exception indicating 401 Unauthorized. This happens in our development environment with IIS Express. There aren't any redirects happening.

We've not been able to find a suitable workaround. We need to allow both windows authentication and anonymous access to the site.

Here is the SO post that my team member posted a few days ago trying to figure out what's wrong: https://stackoverflow.com/q/44396390/2124219

Would really appreciate any help!

Thanks!

[davidsh]

@karelz I've run into this issue. We have ported a Windows 8.1 store app to UWP and the same code that worked in Windows 8.1 doesn't work in UWP.

This is very interesting. This issue here was about .NET Core only and not any UWP scenarios. Thus, it was presumed to be a bug in our WinHttpHandler stack which is used on .NET Core.

But UWP uses a completely different HTTP stack for the System.Net.Http.* implementation. We have not had any reports of authentication problems with UWP apps.

Can you post a small repro of this UWP scenario? Please include a complete Visual Studio solution for the UWP app and, separately, if possible, a small repro for the server as well.

[sbraswell]

@davidsh After some more diligent troubleshooting we've not been able to prove that we're having the same issue as described here. Sorry for the distraction :(

What we've learned is that using the Windows.Net.Http namespace or the System.Net.Http namespace in our UWP app causes the same behavior. My assumption that order of the WWW-Authenticate headers mattered was incorrect. We've determined that unless the URL of our web service is listed in the "Intranet Sites" in IE, the NTLM authentication will never happen. If our URL is listed as an intranet site then NTLM authentication happens correctly. We still don't understand why this was changed from the Windows 8.1 app to the Windows 10 UWP app and would like to return to the behavior found in Windows 8.1.

Thanks for responding so quickly and again, sorry for the distraction with my unrelated issue.

[davidsh]

We've determined that unless the URL of our web service is listed in the "Intranet Sites" in IE, the NTLM authentication will never happen. If our URL is listed as an intranet site then NTLM authentication happens correctly. We still don't understand why this was changed from the Windows 8.1 app to the Windows 10 UWP app and would like to return to the behavior found in Windows 8.1.

Windows 8/8.1 Store Apps used the full .NET Framework. That HTTP stack did not have a requirement that the service be considered "Intranet". Windows 10 Store Apps (UWP) use the .NET Core stack and not full .NET Framework. And in particular, System.Net.Http for .NET Core on UWP uses the WinRT WIndows.Web.Http APIs.

The underlying Windows.Web.Http APIs use an HTTP stack that requires the endpoint be considered as "Intranet" before default credentials will be released over the network. An endpoint can be considered "Intranet" on its own if it is in the same subnet and uses single label DNS names, for example. Otherwise, it must be explicitly listed in the Intranet zone by using the IE "Intranet Sites" tool.

And in addition, any Windows Store Apps that uses enterprise credentials (default credentials for example or explicit domain credentials) must have the Enterprise Authentication capability.

These changes in behavior from Windows 8/8.1 Store Apps to Windows 10 UWP App are expected behavior changes and help improve app security.

[sbraswell]

Thanks for clarifying the distinction and giving context for the behavior change. I'll share this information with the rest of my team. I really appreciate your help. We'll take these changes into consideration as we move forward with our solution development. Thanks again!

[seriouz]

I don't get it to work. The server always returns a 401. (OWA)

                var credential = new NetworkCredential("Peter", "yc3X1f&Ci+ew");
                var credCache = new CredentialCache();
                credCache.Add(new Uri("https://10.0.0.111/"), "Bearer", credential);
                credCache.Add(new Uri("https://10.0.0.111/"), "Basic realm=\"10.0.0.111\"", credential);
                credCache.Add(new Uri("https://10.0.0.111/"), "NTLM", credential);
                credCache.Add(new Uri("https://10.0.0.111/"), "NEGOTIATE", credential);

                var httpHandler = new HttpClientHandler();
                httpHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, errors) => { return true; };
                httpHandler.AllowAutoRedirect = true;
                httpHandler.Credentials = credCache;
                httpHandler.PreAuthenticate = true;
                httpHandler.UseDefaultCredentials = false;
                httpHandler.UseProxy = false;
                httpHandler.UseCookies = true;
                httpHandler.CookieContainer = new CookieContainer();

                var httpClient = new HttpClient(httpHandler);
                var response = await httpClient.GetAsync(new Uri("https://10.0.0.111/ews//exchange.asmx/....."));

[davidsh]

I don't get it to work. The server always returns a 401. (OWA)

Can you please attach a full Visual Studio solution of the repro? Otherwise it is not possible for us to troubleshoot this. Also, what kind of application are you running? A netcoreapp? A Windows UWP store app? Can you also attach a Fiddler trace and/or wireshark trace?

[seriouz]

Here is the code of the not working example WebGet.zip

1. GET

   GET https://10.0.0.111/ews/exchange.asmx/s/GetUserPhoto?email=user@domain.com&size=HR240x240 HTTP/1.1
   Connection: Keep-Alive
   Accept-Encoding: gzip, deflate
   Host: 10.0.0.111

   Response

   HTTP/1.1 401 Unauthorized
   Server: Microsoft-IIS/8.5
   request-id: 065ae685-7dba-4247-8fd5-93fedf2c4546
   X-WSSecurity-Enabled: True
   X-WSSecurity-For: None
   X-OAuth-Enabled: True
   WWW-Authenticate: Negotiate
   WWW-Authenticate: NTLM
   X-Powered-By: ASP.NET
   X-FEServer: DAG2
   Date: Mon, 07 Aug 2017 10:09:40 GMT
   Content-Length: 0
   Proxy-Support: Session-Based-Authentication

2. GET

   GET https://10.0.0.111/ews/exchange.asmx/s/GetUserPhoto?email=user@domain.com&size=HR240x240 HTTP/1.1
   Connection: Keep-Alive
   Accept-Encoding: gzip, deflate
   Host: 10.0.0.111
   Authorization: Negotiate TlRMTVNTUAABAAAAl4II4gAAAAAAAAAAAAAAAAAAAAAKANc6AAAADw==

   Response

   HTTP/1.1 401 Unauthorized
   Server: Microsoft-IIS/8.5
   request-id: 7c6df650-f580-4e33-bb19-1341db6274e0
   WWW-Authenticate: Negotiate TlRMTVNTUAACAAAADgAOADgAAAAVgoni1PFaJUgzLXoAAAAAAAAAAJgAmABGAAAABgOAJQAAAA9GAEEASQBSAFQARQBDAAIADgBGAEEASQBSAFQARQBDAAEACABEAEEARwAyAAQAHABmAGEAaQByAHQAZQBjAC4AaQBuAHQAZQByAG4AAwAmAGQAYQBnADIALgBmAGEAaQByAHQAZQBjAC4AaQBuAHQAZQByAG4ABQAcAGYAYQBpAHIAdABlAGMALgBpAG4AdABlAHIAbgAHAAgAh/OlSGUP0wEAAAAA
   WWW-Authenticate: NTLM
   X-Powered-By: ASP.NET
   X-FEServer: DAG2
   Date: Mon, 07 Aug 2017 10:09:40 GMT
   Content-Length: 0
   Proxy-Support: Session-Based-Authentication

3. GET

   GET https://10.0.0.111/ews/exchange.asmx/s/GetUserPhoto?email=user@domain.com&size=HR240x240 HTTP/1.1
   Connection: Keep-Alive
   Accept-Encoding: gzip, deflate
   Host: 10.0.0.111
   Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAJoAAAA+AT4BsgAAACAAIABYAAAABAAEAHgAAAAeAB4AfAAAABAAEADwAQAAFYKI4goA1zoAAAAPqtTE2yovEtzG/08HxuLrMk0AaQBjAHIAbwBzAG8AZgB0AEEAYwBjAG8AdQBuAHQAagBtAEYAQQAtAEgAQgAtAEoATQAtAFMAVQBSAEYAQQBDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAGrE1tpRFsN9k+1vbXciYBAQAAAAAAAIfzpUhlD9MBujhBUL3uVL4AAAAAAgAOAEYAQQBJAFIAVABFAEMAAQAIAEQAQQBHADIABAAcAGYAYQBpAHIAdABlAGMALgBpAG4AdABlAHIAbgADACYAZABhAGcAMgAuAGYAYQBpAHIAdABlAGMALgBpAG4AdABlAHIAbgAFABwAZgBhAGkAcgB0AGUAYwAuAGkAbgB0AGUAcgBuAAcACACH86VIZQ/TAQYABAACAAAACAAwADAAAAAAAAAAAQAAAAAgAAB7U57jBiBiwF9954KrGdBl2nlCvSch2C9N/j7YfojKBwoAEABbJPatdbojdVYHptMJZE7gCQAeAEgAVABUAFAALwAxADAALgAwAC4AMAAuADEAMwA0AAAAAAAAAAAAAAAAAH3Lq9d6REePGuwaAERGFyU=

   Response

   HTTP/1.1 401 Unauthorized
   Server: Microsoft-IIS/8.5
   request-id: 8918172f-ddd3-4637-9ca4-81127162ec58
   WWW-Authenticate: Negotiate
   WWW-Authenticate: NTLM
   X-Powered-By: ASP.NET
   X-FEServer: DAG2
   Date: Mon, 07 Aug 2017 10:09:40 GMT
   Content-Length: 0
   Proxy-Support: Session-Based-Authentication

Edit: When i copy the source from the dotnet core project into a .net46 project every things works out of the box.

[WallaceKelly]

I too have source code that authenticates when net45 is targeted, but not when netcoreapp2.0 is targeted.

See https://stackoverflow.com/questions/46306508/httpclient-usedefaultcredentials-windows-authentication-net-core-2-0-console

(No, I have not tried to examine the order of the WWW-Authenticate headers.)

[sevaa]

Same issue here. The response headers go:

WWW-Authenticate: Bearer WWW-Authenticate: Basic realm="http://tfsserver:8080/tfs" WWW-Authenticate: NTLM

The WinHTTP-based client fails to send the current Windows credentials, instead fails with error 401.

The server is TFS 2017u2. The client is a WSH Javascript, which goes:

var Req = new ActiveXObject("WinHttp.WinHttpRequest.5.1");
Req.Open("GET", "http://tfsserver:8080/tfs/MyCollection/_apis/projects", false);
Req.SetAutoLogonPolicy(0);
Req.Send();

WScript.Echo(Req.Status);
WScript.Echo(Req.ResponseText);

Specifically for TFS 2017+, a workaround exists: get a Personal Access Token (PAT) from the Web UI, then use basic auth with blank username and PAT for the password.

[rmkerr]

I'm able to repro this issue and will be looking into what's going on. Currently I'm comparing the behavior between .Net Framework and .Net Core, and it looks like things are substantially different. Once I've gotten a chance to look into it more I'll update with details on the behavior difference. Interestingly, I'm only able to repro in the situation described by the original post where the WWW-Authenticate: NTLM header is preceded by a WWW-Authenticate: Basic header. When I change the first header to WWW-Authenticate: Negotiate as described in several of the posts above, I am able to authenticate as expected.

[rmkerr]

So, here's the behavior I'm seeing when targeting .NET Core and modifying the headers with fiddler. I've gone ahead and cut out the bodies of the responses in order to condense things. Notice how in the 401 responses the NTLM header is preceded by a Bearer header. The behavior we're about to see also occurs when I replace that first header with a Basic header, but not with a Negotiate header.

Client:

GET http://mytfsserver/project1/_projects HTTP/1.1
Proxy-Connection: Keep-Alive
Host: mytfsserver

Server:

HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/10.0
X-TFS-ProcessId: ae5e59c3-6ee6-4057-bdea-977ac3f869ba
ActivityId: 56f78e58-bd6c-4535-bcc3-0d483b429468
X-TFS-Session: 56f78e58-bd6c-4535-bcc3-0d483b429468
X-VSS-E2EID: 56f78e58-bd6c-4535-bcc3-0d483b429468
X-FRAME-OPTIONS: SAMEORIGIN
X-TFS-SoapException: %3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22utf-8%22%3F%3E%3Csoap%3AEnvelope%20xmlns%3Asoap%3D%22http%3A%2F%2Fwww.w3.org%2F2003%2F05%2Fsoap-envelope%22%3E%3Csoap%3ABody%3E%3Csoap%3AFault%3E%3Csoap%3ACode%3E%3Csoap%3AValue%3Esoap%3AReceiver%3C%2Fsoap%3AValue%3E%3Csoap%3ASubcode%3E%3Csoap%3AValue%3EUnauthorizedRequestException%3C%2Fsoap%3AValue%3E%3C%2Fsoap%3ASubcode%3E%3C%2Fsoap%3ACode%3E%3Csoap%3AReason%3E%3Csoap%3AText%20xml%3Alang%3D%22en%22%3ETF400813%3A%20Resource%20not%20available%20for%20anonymous%20access.%20Client%20authentication%20required.%3C%2Fsoap%3AText%3E%3C%2Fsoap%3AReason%3E%3C%2Fsoap%3AFault%3E%3C%2Fsoap%3ABody%3E%3C%2Fsoap%3AEnvelope%3E
X-TFS-ServiceError: TF400813%3A%20Resource%20not%20available%20for%20anonymous%20access.%20Client%20authentication%20required.
X-Powered-By: ASP.NET
P3P: CP="CAO DSP COR ADMa DEV CONo TELo CUR PSA PSD TAI IVDo OUR SAMi BUS DEM NAV STA UNI COM INT PHY ONL FIN PUR LOC CNT"
Lfs-Authenticate: NTLM
X-Content-Type-Options: nosniff
Date: Thu, 01 Feb 2018 00:59:48 GMT
Content-Length: 20088
WWW-Authenticate: Bearer
WWW-Authenticate: NTLM

Client:

GET http://mytfsserver/project1/_projects HTTP/1.1
Proxy-Connection: Keep-Alive
Host: mytfsserver

Server:

HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/10.0
X-TFS-ProcessId: ae5e59c3-6ee6-4057-bdea-977ac3f869ba
ActivityId: 56f78e5a-bd6c-4535-bcc3-0d483b429468
X-TFS-Session: 56f78e5a-bd6c-4535-bcc3-0d483b429468
X-VSS-E2EID: 56f78e5a-bd6c-4535-bcc3-0d483b429468
X-FRAME-OPTIONS: SAMEORIGIN
X-TFS-SoapException: %3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22utf-8%22%3F%3E%3Csoap%3AEnvelope%20xmlns%3Asoap%3D%22http%3A%2F%2Fwww.w3.org%2F2003%2F05%2Fsoap-envelope%22%3E%3Csoap%3ABody%3E%3Csoap%3AFault%3E%3Csoap%3ACode%3E%3Csoap%3AValue%3Esoap%3AReceiver%3C%2Fsoap%3AValue%3E%3Csoap%3ASubcode%3E%3Csoap%3AValue%3EUnauthorizedRequestException%3C%2Fsoap%3AValue%3E%3C%2Fsoap%3ASubcode%3E%3C%2Fsoap%3ACode%3E%3Csoap%3AReason%3E%3Csoap%3AText%20xml%3Alang%3D%22en%22%3ETF400813%3A%20Resource%20not%20available%20for%20anonymous%20access.%20Client%20authentication%20required.%3C%2Fsoap%3AText%3E%3C%2Fsoap%3AReason%3E%3C%2Fsoap%3AFault%3E%3C%2Fsoap%3ABody%3E%3C%2Fsoap%3AEnvelope%3E
X-TFS-ServiceError: TF400813%3A%20Resource%20not%20available%20for%20anonymous%20access.%20Client%20authentication%20required.
X-Powered-By: ASP.NET
P3P: CP="CAO DSP COR ADMa DEV CONo TELo CUR PSA PSD TAI IVDo OUR SAMi BUS DEM NAV STA UNI COM INT PHY ONL FIN PUR LOC CNT"
Lfs-Authenticate: NTLM
X-Content-Type-Options: nosniff
Date: Thu, 01 Feb 2018 00:59:51 GMT
Content-Length: 20088
WWW-Authenticate: Bearer
WWW-Authenticate: NTLM

Connection Closed

Since there are no redirects happening here I do not think that the issue is with how we clear credentials on redirect. I've stepped through the code, and I can see that in WinHttpHandler we are properly setting the WINHTTP_OPTION_AUTOLOGON_POLICY to WINHTTP_AUTOLOGON_SECURITY_LEVEL_LOW, and so WinHttp should be sending the default credentials. As you can see in the examples above, it isn't. From here, I'll be trying to answer the following questions:

1. Where, if anywhere, are we re-configuring WinHttp in such a way that it will not send default credentials?
2. Where, if anywhere, do we have behavior that would depend on the order of WWW-Authenticate headers?

If I both of those turn into dead ends, I'll try to reproduce the issue with just WinHttp.

[rmkerr]

There are a lot of very different situations described in this thread, so I'm only going to be touching on the original. After thoroughly testing things on my machine, I believe that the behavior seen in the original report is the result of a bug in the Fiddler proxy rather than in .Net Core.

The questionable behavior is caused by the Proxy-Support: Session-Based-Authentication header. Since NTLM is session based, it requires the Proxy-Support: Session-Based-Authentication header to be included when authenticating through a proxy. Fiddler appears to add that header only if the first WWW-Authenticate header in the 401 response is either NTLM or Negotiate, or if a new connection is created and preauthenticated (The case when we receive a 401 in .Net Framework. This is a behavior difference but I think it is unlikely to change). Since the first request meets neither of those criteria, Fiddler will not include the Proxy-Support: Session-Based-Authentication header. Without that header, NTLM cannot be safely used through a proxy, so the Authorization header is not sent and authentication does not succeed.

If it is possible to reproduce this issue without a proxy involved, I would be really interested in seeing network traces of the issue occurring. Otherwise I think we can close this issue.

[rmkerr]

Since this bug appears to be a result of the tools used to observe it, I'm going to close this issue. If anyone is able to recreate the issue without fiddler or another proxy and can provide a repro, do not hesitate to re-open this issue.

[karelz]

@rmkerr did you report it to Fiddler?

[seriouz]

I don't think its a Fiddler issue. I still believe it is a .net core issue. Look at my post. There is a little tool which does not work under net core but on full framework.

[rmkerr]

@karelz I have not yet, but I found the correct place to report it and am sending in a bug report now.

@seriouz I was only able to reproduce the issue when going through Fiddler, even using the repro you provided above. If you can provide more details or even better a full repro with either a server endpoint or a setup in a VM that would be fantastic.

[seriouz]

@rmkerr I will report here, when i've got more time to investigate again. I'll repost here when i have news. 😃

[dbrownxc]

.NET Core 2.0. I don't know if what I'm seeing is this issue or a related issue. I have a .Net Core app using RestSharp but I also have a set of unit tests with stripped down code. I've reproduced the issue with both .NET Core on Windows and .NET Core on CentOs VM using microsoft/aspnetcore-build:2.0 image as a base (same Windows host) in the tests. Issue is present in HttpClient AND WebRequest. All of this is verified using Wireshark and without Wireshark.

In my scenario, NetworkCredentials(user,pass) doesn't work on Linux. So I started using CredentialCache and Add(uri, authType, NetworkCredentials(user,pass)). I can get this to work in both windows and linux, but only if I used different auth types!

For windows Negotiate works, but NTLM doesn't start the handshake. For linux vm in a docker build NTLM works, but Negotiate doesn't start the handshake.

Here is the server's challenge header

HTTP/1.1 401 Unauthorized
Content-Type: text/html
Server: Microsoft-IIS/8.5
WWW-Authenticate: NTLM
WWW-Authenticate: Negotiate
X-Powered-By: ASP.NET
Date: Thu, 01 Mar 2018 22:17:34 GMT
Content-Length: 1293

HttpClient

var creds = new CredentialCache();
creds.Add(new Uri(addy),"NTLM",new NetworkCredential(Username,Password));
var handler = new HttpClientHandler
{
    Credentials = creds,
};

HttpClient client = new HttpClient(handler);
client.BaseAddress = new Uri(addy);

var response = await client.GetAsync("api/myresource");

WebRequest

var creds = new CredentialCache();
creds.Add(new Uri(addy), "Negotiate", new NetworkCredential(Username, Password));

var client = WebRequest.Create(addy);
client.Credentials = creds;
var response = await client.GetResponseAsync()

I tried overloading it by including both in the cache but that doesn't work. Also, not running Fiddler.

[luvhsail]

hi @davidsh , @seriouz . I am facing the same issue. The code given below works fine in a console app but does not work in UWP. I am not having Fall Creator's update yet, is that related? Do you guys have some work around.

public string GetUsingHttpClient() { string response = string.Empty; try { HttpClientHandler ch = new HttpClientHandler(); ch.UseDefaultCredentials = true; using (var httpClient = new HttpClient(ch)) { httpClient.DefaultRequestHeaders.Add("User-Agent", @"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome / 58.0.3029.110 Safari / 537.36"); response = httpClient.GetStringAsync(new Uri(@"http://localhost:6065/api/user")).Result; } } catch (Exception ex) { response = ex.Message + ex.StackTrace; } return response; }

[seriouz]

My workaround is currently having an external program compiled with dotnet fullframework 4.6 running on a windows machine working as a service for my main dotnet core application running on a linux server.

[rmkerr]

Reopening this as it does seem to be an issue. I'll take a look at the repros you've included later today!