UseManagedNtlm in NET8 cannot authenticate NTLM on IIS with Extended Protection Accept & Required

[weedcry]

Description

I have the same error issue with this issue #19366 And I used UseManagedNtlm in the csproj file to solve it but I encountered another problem when using UseManagedNtlm on NET 8.

On ews on-premise with IIS, windows extended protection (WEP) have 3 level include Off, Accept, Required.

At NET 7, i have successfully authenticated with NTLM authentication when setting all 3 levels on IIS Server

After upgrading to NET8 with config

<ItemGroup>
    <RuntimeHostConfigurationOption Include="System.Net.Security.UseManagedNtlm" Value="true" />
</ItemGroup>

I only successfully authenticated with NTLM authentication when setting level 1 (Off) with level 2 & 3 (Accept, Required) when I perform NTLM authentication, I receive an error message that is "Unauthorized" I want to be able to successfully authenticate at level 2 & 3 on IIS Server, what should I do?

Reproduction Steps

Send request NTLM authenticate to server ews on-premise

Expected behavior

none

Actual behavior

none

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

Project sample: SocketRequestSample-Net8.zip

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones See info in area-owners.md if you want to be subscribed.

Issue Details

---

### Description I have the same error issue with this issue [UseManagedNtlm](https://github.com/xamarin/xamarin-macios/issues/19366) And I used UseManagedNtlm in the csproj file to solve it but I encountered another problem when using UseManagedNtlm on NET 8. On ews on-premise with IIS, windows extended protection (WEP) have 3 level include Off, Accept, Required.  At NET 7, i have successfully authenticated with NTLM authentication when setting all 3 levels on IIS Server After upgrading to NET8 with config ``` ``` I only successfully authenticated with NTLM authentication when setting level 1 (Off) with level 2 & 3 (Accept, Required) when I perform NTLM authentication, I receive an error message that is "Unauthorized" I want to be able to successfully authenticate at level 2 & 3 on IIS Server, what should I do? ### Reproduction Steps Send request NTLM authenticate to server ews on-premise ### Expected behavior none ### Actual behavior none ### Regression? _No response_ ### Known Workarounds _No response_ ### Configuration _No response_ ### Other information Project sample: [SocketRequestSample-Net8.zip](https://github.com/dotnet/runtime/files/13598496/SocketRequestSample-Net8.zip)

Author:	weedcry
Assignees:	-
Labels:	`area-System.Net.Security`, `untriaged`
Milestone:	-

[wfurt]

Did you use managed NTLM in 7 @weedcry? And for 8, what makes you to use it? Could this be related to token binding @filipnavara?

[filipnavara]

And for 8, what makes you to use it?

Same question here. The default in .NET 7 and 8 seems to be the same. If it worked for you, why change it?

Furthermore, on my project we connect to EWS on iOS and we have used NSUrlSessionHandler instead of SocketsHttpHandler precisely due to all the issues with NTLM. It has its own NTLM implementation and it's the same one that Apple uses in the Mail app.

Could this be related to token binding @filipnavara?

Yes, seems so. That said, token binding in general should be implemented.

• The binding token is passed in SocketsHttpHandler from the underlying connection to NegotiateAuthentication here
• SslStream seems to implement ChannelBindingKind.Endpoint here
• Managed NTLM writes the binding here and here

The fact that even the Accept option fails suggests that a wrong value is written in there instead of writing no value at all.

[filipnavara]

I found some of the channel binding code that looks fishy:

https://github.com/dotnet/runtime/blob/b2f371df8f4683b2063f06a30cf644c22bbc4cd1/src/libraries/System.Net.Security/src/System/Net/Security/Pal.Managed/SafeChannelBindingHandle.cs#L21-L44

The first constructor is the one used on macOS/iOS with the Pal.Managed implementation. Compared to the other one it's missing the prefix.

Nevermind, I misread that.

[weedcry]

@wfurt @filipnavara Thank you for your response.

Did you use managed NTLM in 7? And for 8, what makes you use it?

In Net 7, I did not use managed NTLM but just only used SocketsHttpHandler to send requests. Although I could authenticate NTLM with all 3 levels on the IIS Server (Off, Accept, Required), but I could not authenticate NTLM with the username format user@domain.com as in issue #94303.

I was forced to upgrade the system to NET 8 and use the configuration UseManagedNtlm and I encountered the error as described in this issue.

The reason I used SocketsHttpHandler instead of NSUrlSessionHandler on iOS is due to the server proxy issue. I wanted to be able to constantly change the server proxy in the application for the outgoing request, so I chose SocketsHttpHandler. And to be able to authenticate NTLM with the username format user@domain.com, I used UseManagedNtlm on NET 8.

[vinhdp195]

@filipnavara

Same question here. The default in .NET 7 and 8 seems to be the same. If it worked for you, why change it? Furthermore, on my project we connect to EWS on iOS and we have used NSUrlSessionHandler instead of SocketsHttpHandler precisely due to all the issues with NTLM.

I use SocketsHttpHandler instead of NSUrlSessionHandler because on the iOS platform, NSUrlSessionHandler does not support custom proxies. https://github.com/xamarin/xamarin-macios/blob/xamarin.ios-15.10.0.1/src/Foundation/NSUrlSessionHandler.cs#L625

        // We dont support any custom proxies, and don't let anybody wonder why their proxy isn't
        // being used if they try to assign one (in any case we also return false from 'SupportsProxy').
        [UnsupportedOSPlatform ("ios")]
        [UnsupportedOSPlatform ("maccatalyst")]
        [UnsupportedOSPlatform ("tvos")]
        [UnsupportedOSPlatform ("macos")]
        public IWebProxy? Proxy {
        ...
        }

[weedcry]

@filipnavara I have tried using NSUrlSessionHandler instead of SocketsHttpHandler as you suggested. But currently, I can only authenticate NTLM at the Off & Accept option. When I setting the Required option, NSUrlSessionHandler fails to authenticate with the returned message being “Unauthorized”

[filipnavara]

Thanks for the additional testing. I plan to look into this in more detail, but I have been preoccupied with other issues until now.

[weedcry]

@filipnavara Thank you. I look forward to your support.

[filipnavara]

I think I found a flaw in how the channel bindings hash is calculated. It's supposed to hash over the GSSAPI octet version of the binding which means we need to add some padding and length prefixes.

[SteveSyfuhs]

Also, a proxy is potentially involved.

The reason I used SocketsHttpHandler instead of NSUrlSessionHandler on iOS is due to the server proxy issue. I wanted to be able to constantly change the server proxy in the application for the outgoing request, so I chose SocketsHttpHandler.

That isn't usually compatible with such binding mechanisms.

[weedcry]

@SteveSyfuhs I also tried the case without using a proxy. For SocketsHttpHandler, it can only authenticate NTLM with the Off option. As for NSUrlSessionHandler, it can authenticate with both Off & Accept options. For the Required option, NTLM authentication is still not possible. According to what filipnavara said, it seems the problem lies in dotnet.

[filipnavara]

Steve is right that extended authentication and some types of proxies don’t mix well. The point of the extended authentication is to prevent man-in-the-middle attack by binding the authentication to some TLS token of the HTTPS transport. It may work if the TLS connections are pass-through but it certainly won’t work for intercepting proxies. I specifically tested the intercepting proxy scenario with Fiddler and the authentication is (correctly) rejected even after the fix is applied.

[weedcry]

@filipnavara So, this issue will be resolved in NET9, right? If it’s NET9, I think the release time will be quite long, so my current solution for NET8 is to use NSUrlSessionHandler to authenticate NTLM at the Accept option. With NSUrlSessionHandler, I plan to use this method to connect to the proxy server 14632 Do you think it will be okay? While NSUrlSessionHandler on iOS does not support proxy servers, I haven’t found any documentation that clearly explains this issue.

[rzikm]

.NET networking team member here (filipnavara is an external contributor)

So, this issue will be resolved in NET9, right?

Yes.

Given that the fix is not so complex and it is a regression from .NET 7.0, we would not object to porting it to .NET 8 (earliest merge window would be for February servicing release).

@weedcry Can you provide us some information on how much does this issue impact your business? Is not using managed NTLM possible workaround for you? If we provide you private build of the updated .NET 8 binaries, would you be able to test if the change fixes your problem?

[filipnavara]

it is a regression from .NET 7.0

Technically it's not a regression. The same problem exists on .NET 7 but managed NTLM is used only for Android there. On .NET 8 you have to opt-in with an undocumented switch on any non-Android platform.

That said, backport would be nice since .NET 8 is LTS and the fix is quite small and self-contained.

[vinhdp195]

Here are the options:

1/ We use SocketsHttpHandler, with using managed NTLM set to TRUE, and pass in a WebProxy as a parameter ➜ However, this option is encountering this issue, it cannot authenticate with WEP config set to Allow or Required

2/ We use SocketsHttpHandler, without using managed NTLM, and pass in a WebProxy as a parameter ➜ This option is encountering an issue here https://github.com/dotnet/runtime/issues/94303, it cannot authenticate with UPN

3/ We use NSUrlSessionHandler, without using managed NTLM ➜ This option can solve (2), however, it still cannot authenticate with WEP config set to Required Especially for option (3), NSUrlSessionHandler does not support passing in a WebProxy (here https://github.com/xamarin/xamarin-macios/issues/14632)

Comments: For (1), I need to wait for the next .NET release, so it’s a long wait Therefore, I am trying to use option (3) by passing in Proxies using the method here: https://github.com/xamarin/xamarin-macios/issues/14632#issuecomment-1853260766 @rzikm do you think the method of passing in Proxies as above is good? I don’t see any documentation talking about this?

[rzikm]

@rzikm do you think the method of passing in Proxies as above is good? I don’t see any documentation talking about this?

I am not familiar with NSUrlSessionHandler so I am unable to comment on that. That class belongs to the xamarin repository so their developers need to answer the question there.

@karelz would the comment above be sufficient justification for backport?

[weedcry]

@rzikm

Can you provide us some information on how much does this issue impact your business

The impact is significant on my business. All users are unable to authenticate NTLM to the on-premise ews server to get information.

Is not using managed NTLM possible workaround for you?

Users need to authenticate with the username format user@domain.com. The solution to use this format is using managed NTLM We can use solution number 3 above from @vinhdp195 as a temporary replacement, but it still can’t authenticate the Required Option.

If we provide you private build of the updated .NET 8 binaries, would you be able to test if the change fixes your problem?

Could you provide me with a private build of the updated .NET 8 binaries for me Let me check if it fixed my problem or not Thank you for your help.

[rzikm]

Could you provide me with a private build of the updated .NET 8 binaries for me

See attached dll Debug.zip

To use it, publish your app as self-contained, and replace the System.Net.Security.dll with the one from the zip (You did not mention which platform you are running so the zip includes more versions). Let me know if it works.

[weedcry]

@rzikm My app is running on the iOS platform (net8.0-ios).

See attached dll Debug.zip

In the net8.0-ios folder, I only see files in json, pdb, xml formats. Could you guide me on how to replace it for System.Net.Security.dll ? I think it would be easier to replace if you could provide me with a dll format file.

[rzikm]

@weedcry Are you sure your antivirus program or something is not interfering? I just checked by downloading the file back and the dll is present

That being said, I don't know how to test private bits on non-desktop platforms. maybe @simonrozsival can help here.

[weedcry]

@rzikm I apologize for this issue. I have checked again. The reason is that my antivirus program automatically deleted it when I unzipped it. Thank you for your help.

[weedcry]

@rzikm I have implemented the System.Net.Security.dll (modified) in my iOS application. It is now possible to use UseManagedNtlm on SocketsHttpHandler to send NTLM authentication requests with the options: Off, Accept and Required. I think it has been able to solve this issue.

[karelz]

I am a bit confused here, so apologies for stupid questions:

@filipnavara I think you might be able to answer most of them - I would appreciate that if you can, thanks!

1. UseManagedNtlm is enabled ONLY for Android on .NET 8+
2. For non-Android platforms it is UNDOCUMENTED opt-in on .NET 8+ ... therefore it is unsupported
   • I assume it is opt-in for non-Android due to lack of testing and/or known problems, right @filipnavara?
3. The problem in this issue was reported against iOS, not against Android if I understand it correctly.

As a result, I don't think we can make the case to service it for iOS. If it was problem also for Android users, it would mean that a new feature is not working completely and it might have a chance to be backported to servicing (although no promises - I am not the decision maker). It would depend on workarounds -- for example, why isn't NSUrlSessionHandler working properly with proxies and if that should be fixed first.

@steveisok do you have different opinion or insight from Android/iOS perspective?

[filipnavara]

1. UseManagedNtlm is enabled ONLY for Android on .NET 8+

Correct. By default it's enabled only for Android on .NET 8.

2. For non-Android platforms it is UNDOCUMENTED opt-in on .NET 8+ ... therefore it is unsupported

   • I assume it is opt-in for non-Android due to lack of testing and/or known problems, right @filipnavara?

The code had to be restructured to support the managed NTLM implementation on other Unix-like platforms where it co-exists as an option alongside system GSSAPI backend. The reconstruction happened very late in the .NET 8 release cycle. As a safety precaution the defaults were kept the same as in .NET 7 and some breaking changes landed only for .NET 9 (eg. defaulting to managed NTLM on all Apple platforms).

3. The problem in this issue was reported against iOS, not against Android if I understand it correctly.

It was reported against iOS but the issue is NOT specific to iOS. We at eM Client actually face the same issue on Android since we use the SocketsHttpHandler to connect to Exchange servers.

[karelz]

Fixed in main (9.0) in PR #95898 and in 8.0.7 in PR #102565 (July release).