Preauthentication for Negotiate

[trexx]

A related issue: https://github.com/dotnet/runtime/issues/30195

I have a simple poc which connects to Elasticsearch authenticated by Kerberos. Elasticsearch only supports request based authentication, so each time I need to ship a log, it requires 2 round trips as the first request always returns a 401 unauthorised. It would be useful to use preauthentication to remove that 401 round trip and immediately send the header for applications that don't support session based authentication.

This was the behaviour before SocketHttpHandler (via AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);) as WinHttpHandler was able to preauth a Kerberos connection however when running on Linux we've discovered that the libcurl based handler does not behave the same way.

[davidsh]

What version of .NET Core are you using?

In general, Negotiate/NTLM authentication uses a challenge/response mechanism and authenticates connections and not requests. So, once a TCP connection is reused, the server doesn't send back any 'Www-Authenticate: Negotiate' response header. It considers the connection already authenticated.

So, I'm not sure what "pre-authentication" you are referring to. Authentication mechanisms like Basic and Digest can support pre-authentication on a request since they require the 'Authorization " header to be sent on every request to the server.

[davidsh]

This was the behaviour before SocketHttpHandler (via AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);) as WinHttpHandler was able to preauth a Kerberos connection

WinHttpHandler can't "pre-authenticate" any Negotiate/NTLM connections. The only thing it can do (like other HTTP stacks) is to reuse existing connections that have already been authenticated with Negotiate/NTLM.

[trexx]

We are using v3.1.

In general, Negotiate/NTLM authentication uses a challenge/response mechanism and authenticates connections and not requests. So, once a TCP connection is reused, the server doesn't send back any 'Www-Authenticate: Negotiate' response header. It considers the connection already authenticated.

This isn't the behaviour I am seeing with Elasticsearch and requires the Authorization header on each request. I believe Elasticsearch does not support session-based authentication.

WinHttpHandler can't "pre-authenticate" any Negotiate/NTLM connections. The only thing it can do (like other HTTP stacks) is to reuse existing connections that have already been authenticated with Negotiate/NTLM.

Sorry to further clarify, I'll provide some scenarios which explain what I'm seeing. Each request goes to the same URL and doesn't change. If I use WinHttpHandler and set PreAuthenticate to true, after the first 401 response each request thereafter will contain a Authorization header thus reducing the number of roundtrips to 1. if I use WinHttpHandler and set PreAuthenticate to false, behaviour is identical to SocketHttpHandler with PreAuthenticate set to true or false ; Each request will be missing an Authorization header, which after the 401 response causes the request to be resent with the Authorization header present. Both handlers are reusing their tcp connections.

And so what I'm suggesting is whether its possible to have SocketHttpHandler behave the same as WinHttpHandler when PreAuthenticate is set to true.

I appreciate the replies and I apologise if I'm not clear.

[trexx]

Hi @davidsh. Is there anyway Socket and Win HttpHandlers can behave similarly when it comes to that PreAuthenticate property?

[davidsh]

Hi @davidsh. Is there anyway Socket and Win HttpHandlers can behave similarly when it comes to that PreAuthenticate property?

This isn't the behaviour I am seeing with Elasticsearch and requires the Authorization header on each request. I believe Elasticsearch does not support session-based authentication.

I'm confused about what you're saying. Are you saying that the Elasticsearch server supports Negotiate/NTLM authentication (which is connection based authentication) and yet still requires the final Authorization: NTLM <base64blob> request header to be attached for all subsequent requests on that same connection?

[trexx]

@davidsh Yes thats correct. Elasticsearch exposes a REST API and as its stateless in nature, would require an Authorization request header on each request.

[davidsh]

Elasticsearch exposes a REST API and as its stateless in nature, would require an Authorization request header on each request.

I can totally understand that for authentication schemes like Basic, Digest, OAuth that can work with stateless backends and don't require connection affinity.

However, authentication protocols such as Negotiate or NTLM use challenge-response mechanisms and only work to authenticate a connection itself. Once authenticated, the server will not demand further 'Authorization' request headers by the client.

So, let's circle back to the core issue. Is this issue about Negotiate or NTLM mechanisms? Or some other authentication mechanism? Negotiate or NTLM requires connection affinity.

[mattpwhite]

However, authentication protocols such as Negotiate or NTLM use challenge-response mechanisms and only work to authenticate a connection itself.

The common case for Negotiate is to use Kerberos, and there's no challenge there exactly, not in the sense that NTLM actually generates a challenge with a nonce. There's an initial 401 to the client that serves to tell it that it should use Negotiate. There's nothing in that message that is actually necessary for the client to generate the require Authorization header, and it doesn't contribute to the eventual negotiated security context. If a client knows that the server supports/will require Kerberos, it can potentially elide that first round trip for the 401 and just include the necessary header in its first request. All it needs to generate that header is a Kerberos ticket for the service in question, which it will almost certainly already have cached in any scenario where this happens often enough for performance to matter. I believe that's the optimization being asked about.

Although there can theoretically be multiple 401s required to generate a security context (and are for NTLM, but NTLM is terrible for both security and performance), Kerberos is typically just that one message from the client. In a REST scenario, it's the difference between 2 (HTTP, we'll ignore TCP and TLS) round trips and 1, which could be significant.

[trexx]

@davidsh The issue is about Negotiate, specifically Kerberos.

The WinHttpHandler has this optimisation when the "PreAuthenticate" property is set to true.