HttpClient With Proxy did not handle the Proxy 407 auth flow on Linux OS

[kashti3]

Description

Hi, I`m developing an application which using HttpClient with HttpClientHandler to manage my Http network connection. While setting my WebProxy on the HttpClientHandler I getting errors of 407 although my Proxy credentials are correct.

see a code snip:

foo(Uri? iProxyUri, NetworkCredential iProxyCredentials){
       CredentialCache cache = [];

       _ = cache.TryAdd(iProxyUri, iProxyCredentials);

         HttpClientHandler handler = new()
        {
            AllowAutoRedirect = false, // we want to handle the redirect 
            ServerCertificateCustomValidationCallback = (iRequest,
                                                         _1,
                                                         _2,
                                                         iErrors) => true,
            Proxy = new WebProxy(iProxyUri, false)
            {
                Credentials = iProxyCredentials,
                UseDefaultCredentials = false
            },
            SslProtocols = MySslProtocol,
            UseProxy = true,
            DefaultProxyCredentials = cache,
            Credentials = cache,
            UseDefaultCredentials = false,
            // PreAuthenticate = true
        };
        using HttpClient client = new(handler)
        {
            Timeout = TimeSpan.FromMinutes(5)
        };
}

Reproduction Steps

1. Create HttpClient with HttpClientHandler which contains proxy that require username and password .
2. Make API call to https://example.com

Expected behavior

407 response will be handle by the client handler and credentials should pass to the proxy server

Actual behavior

Received 407 response although client handler contains the proxy credentials

Regression?

Yes, I have an application with more or less the same code that running on .net 6 and I don`t see this issue there

Known Workarounds

No response

Configuration

No response

Other information

No response

[dotnet-policy-service[bot]]

Tagging subscribers to this area: @dotnet/ncl See info in area-owners.md if you want to be subscribed.

[wfurt]

Can you get packet captures @kashti3 ? And can you make sure your 6 app is identical? Ideally isolate small runable repro.

[kashti3]

@wfurt Also the same code is running from windows and I don`t get this issue.

[wfurt]

what do you see with .NET 6? Since there is no attempt to even try, it feels like some problem with the iProxyCredentials. Since the credentials are set on the proxy, assigning to DefaultProxyCredentials should not be needed.

Also CredentialCache.Add expects authType and your cache.TryAdd skips that.

[osherbendavid]

@wfurt our cache.TryAdd tries to add all types

also, as @kashti3 mentioned, we have the same app (.net 8) running on windows under the same policies and using the same proxy with same creds and all works as expected.

on our customer proxy logs we saw that the CONNECT request is being sent without the required creds. we have ran a Curl request from the linux machine to the same url with the same proxy config, and we got a valid response.

BTW, powershell also fails (as expected)

[wfurt]

the fact that we don't even see attempt to authenticate is interesting. The logic is same as on Windows e.g. this suggests that the handler somehow feels it cannot proceed with authentication. You can try very simple app without any custom code and credential cache @osherbendavid. This is going to be difficult IMHO for us to investigate as we may not be able to reproduce it.

[osherbendavid]

@wfurt we have tried running a simple console app just for the sake of the test, but it still fails.

[osherbendavid]

also, just to clarify - the .net 6 app also never worked properly with the proxy on Linux. sorry for the confusion

[wfurt]

what is your base OS @osherbendavid ?

[osherbendavid]

@wfurt Red Hat Enterprise Linux release 8.5 (Ootpa)

[wfurt]

8.5 seems to be EOS. Do you have gssntlmssp package installed? And if not, could you try to add it? It seems like the packages require authentication so it is not easy for me to check. cc @tmds

[osherbendavid]

@wfurt thank you! installing gssntlmssp solved the issue.

[kashti3]

@wfurt Does this package should be install in all Linux distros?

[wfurt]

The proxy server offered Negotiate, NTLM and Basic. The HttpClient should fall-back to Basic (when first two are not available) ... but it does not for whatever reason. Another option probably is subscribing to managed implementation instead of depending on GSSAPI (#101058)

So the reported issue seems like real problem we should look into.

[kashti3]

@wfurt Just to clarify that @osherbendavid monition that the gssntlmssp package solve the issue. It only work with gssntlmssp-0.7.0-6.el8.x86_64.rpm version.

please share with us the official version to be work with.

[wfurt]

any version should work from .NET prospective. However some distributions had issues when transitioned to OpenSSL 3 and/or obsoleted some ciphers that NTLM needs. As I mentioned I don;t have access to payed packages and it seems that the version you use is out of support. Perhaps something you can take up to RedHat.