Bug Report: Authentication Validation Error (GenericFailure) in .NET8

korovindenis commented 2 months ago

Description

When executing the provided code snippet in a .NET 8 project, an HttpRequestException is thrown with the message Authentication validation failed with error - GenericFailure. This issue occurs when attempting to send an HTTP request using HttpClient with Kerberos authentication and utilizing default credentials.

Reproduction Steps

Create a new .NET 8 project.
Add the following code to the Program.cs file:

using System.Net;
class Program
{
    static async Task Main(string[] args)
    {

        var handler = new HttpClientHandler
        {
            UseDefaultCredentials = true,
        };

        using (var client = new HttpClient(handler))
        {
            try
            {
                HttpResponseMessage response = await client.GetAsync("...");
                response.EnsureSuccessStatusCode();

            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }
}

Run the project.

Expected behavior

The HTTP request is successfully sent, and the response is processed without any exceptions.

Actual behavior

System.Net.Http.HttpRequestException: Authentication validation failed with error - GenericFailure.
   at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, Boolean async, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, Boolean async, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Program.Main(String[] args) in C:\...\Program.cs:line 16

Regression?

In .NET 6 this issue does not occur.

Known Workarounds

Changing the HTTP version to HTTP/1.0 by setting client.DefaultRequestVersion = HttpVersion.Version10.
Using .NET 6 where this issue does not occur.

Configuration

.NET SDK Version: .NET8
Operating System: Win10
Server: Using gokrb5 Kerberos library

Other information

The problem may be related to the following code added in .NET 8: https://github.com/dotnet/runtime/blob/eb765b71a4a93e4157cd7e1eaf33995ffc80285c/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/AuthenticationHelper.NtAuth.cs#L208

Additionally, in the gokrb5 library, there is a constant used for the WWW-Authenticate header set to the client upon successful authentication with HTTP code 200 . Could this be related to the issue?

ManickaP commented 2 months ago

@wfurt any thoughts / suggestions?

wfurt commented 2 months ago

This is going to be difficult to investigate IMHO without repro. The interesting part for me is the 1.0 vs 1.1 as workaround. That almost suggests the problem lives in session bases authentication somehow. (that may perhaps be verified with Connection: close header) I would suggest to collect packet capture and internal traces and look for differences to start with.

korovindenis commented 2 months ago

The issue is reproducible even with HTTP version 1.0 when the client includes the header Connection: Keep-Alive. Transitioning to this HTTP version does not seem to be a good solution.
In HTTP version 1.1, setting the header Connection: close does not resolve the issue.
As a workaround, modifying the constant to ade0234568a4209af8bc0280289eca has been effective. This value is derived from RFC 4559.

filipnavara commented 1 month ago

Would it be possible for you to capture the connection trace with something like Fiddler or Proxyman and share it? I'd like to look at the exact authentication headers.

korovindenis commented 1 month ago

@filipnavara fiddler logs: client

GET / HTTP/1.1
Host: xxxxx
User-Agent: Mozilla/5.0 (Windows NT 10.0; Microsoft Windows 10.0.19045)
Accept-Encoding: gzip, deflate, br

server

Encrypted HTTPS traffic flows through this CONNECT tunnel. HTTPS Decryption is enabled in Fiddler, so decrypted sessions running in this tunnel will be shown in the Web Sessions list.

Secure Protocol: Tls12
Cipher: Aes128 128bits
Hash Algorithm: Sha256 ?bits
Key Exchange: ECDHE_RSA (0xae06) 255bits

== Server Certificate ==========
[Subject]
  XXXXXX

[Issuer]
  XXXXX

[Serial Number]
  5200008750F6793F7DC836B065000000008750

[Not Before]
  03.06.2024 18:36:06

[Not After]
  30.11.2024 18:36:06

[Thumbprint]
  AB11E40456F4AEC0B8CF672D24C083FE129C5224

[SubjectAltNames]
*.XXXX

HTTP/1.1 401 Unauthorized
Www-Authenticate: Negotiate
Www-Authenticate: NTLM
Www-Authenticate: Bearer
Www-Authenticate: Basic
X-Xss-Protection: 1; mode=block
Date: Wed, 31 Jul 2024 06:48:20 GMT
Content-Length: 0
Proxy-Support: Session-Based-Authentication

client

GET / HTTP/1.1
Host: xxxxx
User-Agent: Mozilla/5.0 (Windows NT 10.0; Microsoft Windows 10.0.19045)
Accept-Encoding: gzip, deflate, br
Authorization: Negotiate xxxxxxxx==

server

HTTP/1.1 200 OK
Content-Length: 187
Content-Type: application/json
Www-Authenticate: Negotiate oRQwEqADCgEAoQsGCSqGSIb3EgECAg==
X-Xss-Protection: 1; mode=block
Date: Wed, 31 Jul 2024 06:48:20 GMT

But there is an error in the net8 application.

filipnavara commented 1 month ago

I'd need the request with the Authorization header sent by the client, not just the Www-Authenticate responses. Thanks!

korovindenis commented 1 month ago

@filipnavara I have added the logs (in the comment above), please let me know if any additional logs are needed.

filipnavara commented 1 month ago

I have added the logs (in the comment above), please let me know if any additional logs are needed.

xxxxxxxx== doesn't sound like the real value. I understand if you want to keep the token private. I don't need the actual token value anyway.

You can take the value a paste it into https://asn1js.eu/. Then copy the whole subtree by clicking on the root node and choosing Copy subtree and paste that here.

Feel free to redact the OCTET STRING part which contains the actual authentication token. I am only interested in the rest of the structure and the OIDs.

For example, it may look like this (taken from example in MS specification):

Application_0 @0+349 (constructed): (2 elem)
  OBJECT_IDENTIFIER @4+6: 1.3.6.1.5.5.2
  [0] @12+337 (constructed): (1 elem)
    SEQUENCE @16+333 (constructed): (3 elem)
      [0] @20+26 (constructed): (1 elem)
        SEQUENCE @22+24 (constructed): (2 elem)
          OBJECT_IDENTIFIER @24+10: 1.3.6.1.4.1.311.2.2.30
          OBJECT_IDENTIFIER @36+10: 1.3.6.1.4.1.311.2.2.10
      [2] @48+257 (constructed): (1 elem)
        OCTET_STRING @52+254: (254 byte)|4E45474F4558545301000000000000006000000070000000CFFA11765E12599A347D766852BFCE7097458710BB8242B4C7DFBAD2DA897AA311A7D868463430952562DC13C554F2010000000000000000600000000100000000000000000000005C33530DEAF90D4DB2EC4AE3786EC3084E45474F455854530300000001000000400000008E000000CFFA11765E12599A347D766852BFCE705C33530DEAF90D4DB2EC4AE3786EC308400000004E000000304CA04A3048302A80283026312430220603550403131B584D4C50726F766964657220496E7465726D656469617465204341301A80183016311430120603550403130B584D4C50726F7669646572
      [3] @309+42 (constructed): (1 elem)
        SEQUENCE @311+40 (constructed): (1 elem)
          [0] @313+38 (constructed): (1 elem)
            GeneralString @315+36: not_defined_in_RFC4178@please_ignore

The OCTET_STRING line is the exchanged token which I don't need. I need to see the rest of the metadata to figure out which specific protocol is the client trying to negotiate.

korovindenis commented 1 month ago

@filipnavara

ContentInfo [?] Application_0 @0+6024 (constructed): (2 elem)
  contentType ContentType OBJECT_IDENTIFIER @4+6: 1.3.6.1.5.5.2
  content [0] @12+6012 (constructed): (1 elem)
    ANY SEQUENCE @16+6008 (constructed): (2 elem)
      [0] @20+48 (constructed): (1 elem)
        SEQUENCE @22+46 (constructed): (4 elem)
          OBJECT_IDENTIFIER @24+9: 1.2.840.48018.1.2.2
          OBJECT_IDENTIFIER @35+9: 1.2.840.113554.1.2.2
          OBJECT_IDENTIFIER @46+10: 1.3.6.1.4.1.311.2.2.30
          OBJECT_IDENTIFIER @58+10: 1.3.6.1.4.1.311.2.2.10
      [2] @70+5954 (constructed): (1 elem)
        OCTET_STRING @74+5950 (encapsulates): (5950 byte)|XXXXXXXX
          Application_0 @78+5946 (constructed): (3 elem)
            OBJECT_IDENTIFIER @82+9: 1.2.840.113554.1.2.2
            BOOLEAN @93+0: true
            Application_14 @95+5929 (constructed): (1 elem)
              SEQUENCE @99+5925 (constructed): (5 elem)
                [0] @103+3 (constructed): (1 elem)
                  INTEGER @105+1: 5
                [1] @108+3 (constructed): (1 elem)
                  INTEGER @110+1: 14
                [2] @113+7 (constructed): (1 elem)
                  BIT_STRING @115+5: (32 bit)|00100000000000000000000000000000
                [3] @122+5482 (constructed): (1 elem)
                  Application_1 @126+5478 (constructed): (1 elem)
                    SEQUENCE @130+5474 (constructed): (4 elem)
                      [0] @134+3 (constructed): (1 elem)
                        INTEGER @136+1: 5
                      [1] @139+8 (constructed): (1 elem)
                        GeneralString @141+6: AVP.RU
                      [2] @149+37 (constructed): (1 elem)
                        SEQUENCE @151+35 (constructed): (2 elem)
                          [0] @153+3 (constructed): (1 elem)
                            INTEGER @155+1: 2
                          [1] @158+28 (constructed): (1 elem)
                            SEQUENCE @160+26 (constructed): (2 elem)
                              GeneralString @162+4: HTTP
                              GeneralString @168+18: sandbox.sbx.avp.ru
                      [3] @188+5416 (constructed): (1 elem)
                        SEQUENCE @192+5412 (constructed): (3 elem)
                          [0] @196+3 (constructed): (1 elem)
                            INTEGER @198+1: 23
                          [1] @201+3 (constructed): (1 elem)
                            INTEGER @203+1: 4
                          [2] @206+5398 (constructed): (1 elem)
                            OCTET_STRING @210+5394: (5394 byte)|XXXXXXXX
                [4] @5608+416 (constructed): (1 elem)
                  SEQUENCE @5612+412 (constructed): (2 elem)
                    [0] @5616+3 (constructed): (1 elem)
                      INTEGER @5618+1: 23
                    [2] @5621+403 (constructed): (1 elem)
                      OCTET_STRING @5625+399: (399 byte)|XXXXXXXX

filipnavara commented 1 month ago

Thanks, this is helpful. It means the client is opportunistically sending the Kerberos token with OID 1.2.840.113554.1.2.2.

That does match the OID in the server's response:

[1] @0+20 (constructed): (1 elem)
  SEQUENCE @2+18 (constructed): (2 elem)
    [0] @4+3 (constructed): (1 elem)
      ENUMERATED @6+1: 0
    [1] @9+11 (constructed): (1 elem)
      OBJECT_IDENTIFIER @11+9: 1.2.840.113554.1.2.2

I'll need to dive into the (newer) SPNEGO specification to see what are the requirements for MIC (Message Integrity Check). At least the basic construction of the requests and responses look okay.

wfurt commented 1 month ago

Since there seems to be two server constants - one working and one not - we can perhaps look at the difference...?

filipnavara commented 1 month ago

The other one is an unparsable garbage, so not really much to compare :-/

https://github.com/jcmturner/gokrb5/pull/549#issuecomment-2259776324

filipnavara commented 1 month ago

I read the specification (RFC 4178) and the MIC is actually REQUIRED in this case. The rationale is the following:

Initiator (client) most preferred mechanism is 1.2.840.48018.1.2.2 despite the opportunistic token of type 1.2.840.113554.1.2.2. 1.2.840.113554.1.2.2 is listed as the second most preferred mechanism.
Acceptor (server) accepts the opportunistic token and responds with accept-complete and chosen mechanism 1.2.840.113554.1.2.2.
According to section 5 of RFC 4178, the mechlistMIC is OPTIONAL only if the accepted mechanism is the most preferred mechanism of both the initiator and the acceptor. That's not the case here, so mechlistMIC is REQUIRED.

The server is not implementing the specification correctly. It needs to create the last response programmatically instead of using a predefined value, and it needs to compute the message integrity of the client's request and include it in the response.

AraHaan commented 1 month ago

Kind of a strange issue though.

wfurt commented 1 month ago

Kind of a strange issue though.

why? .NET 8 changed to improve security and verify the messages integrity. I should not be constant to start with.

dotnet / runtime