CSOM for Core (Microsoft.SharePointOnline.CSOM) causes Socket Exhaustion

[lengkeng]

Target SharePoint environment

SharePoint Online

What SharePoint development model, framework, SDK or API is this about?

SharePoint CSOM

Developer environment

Windows

What browser(s) / client(s) have you tested

• [ ] 💥 Internet Explorer
• [ ] 💥 Microsoft Edge
• [ ] 💥 Google Chrome
• [ ] 💥 FireFox
• [ ] 💥 Safari
• [ ] mobile (iOS/iPadOS)
• [ ] mobile (Android)
• [X] not applicable
• [ ] other (enter in the "Additional environment details" area below)

Additional environment details

• Microsoft.SharePointOnline.CSOM version 16.1.21909.12000
• See https://www.nuget.org/packages/Microsoft.SharePointOnline.CSOM
• This happens on Windows in AKS but should happen on Linux as well

Describe the bug / error

CSOM Core uses HttpWebRequest, which creates new instances of HttpClient, and could exhausts the available TCP sockets on a system quickly. See references:

• https://github.com/dotnet/runtime/issues/20013
• https://docs.microsoft.com/en-us/dotnet/core/compatibility/networking/6.0/webrequest-deprecated
• https://github.com/dotnet/runtime/blob/418026050ad0e0d08ea60f77302942d7ba019893/src/libraries/System.Net.Requests/src/System/Net/HttpWebRequest.cs#L1480
• SPWebRequestExecutor class

To exacerbate the issue, .Net 5 no longer supports System.Net.ServicePointManager, which used to allow us to set the DefaultConnectionLimit, and HttpClient in .Net 5 allows for int.Max of connections per destination server by default. As a result of this combination, under high volume in .Net 5, we need to be able to limit the number of connections per destination server. In .Net 5 and .Net 6, the recommended way to do this is by passing in a SocketsHttpHandler or an HttpClient.

Steps to reproduce

1. Making a ton of CSOM calls over a period of time
2. Run netstat -ano and inspect the number of TCP connections in TIME_AWAIT

Expected behavior

CSOM core to use HttpClient and allow caller to pass in a HttpMessageHandler (such as SocketsHttpHandler) instead of HttpWebRequest, as recommended in documentation https://docs.microsoft.com/en-us/dotnet/core/compatibility/networking/6.0/webrequest-deprecated

[ghost]

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

[patrikhellgren]

@lengkeng There is a SPWebRequestExecutor that uses HttpClient available in PnP.Framework and this can be used instead of the default SPWebRequestExecutor in CSOM. You can use the static HttpClient available in PnP.Framework (like in the example) or you can provide your own. See pnp/pnpframework#261 for more info.

Usage example:

var authManager = new PnP.Framework.AuthenticationManager(clientId, certificate, tenantId);
var clientContext = authManager.GetContext(siteUrl);
clientContext.WebRequestExecutorFactory = new HttpClientWebRequestExecutorFactory(PnPHttpClient.Instance.GetHttpClient());
clientContext.Load(clientContext.Web, w => w.Title);
await clientContext.ExecuteQueryRetryAsync();

[AJIXuMuK]

@lengkeng - will @patrikhellgren approach work for you?

[lengkeng]

Hey sorry for the late response. Thank you for the suggestion @patrikhellgren We have been looking into this option. I will update here as soon as we have a result.

[lengkeng]

Folks, we ended up cloning just one file - HttpClientWebRequestExecutorFactory. It ran locally fine on .net 48 but we are seeing an issue running on .net 5 in AKS. Unfortunately I don't have the details but just want to update this thread before I go OOO for a few days. I will update again when we have more info.