pnp / pnpframework

PnP Framework is a .NET library targeting Microsoft 365 containing the PnP Provisioning engine and a ton of other useful extensions
https://pnp.github.io/pnpframework/
MIT License
200 stars 140 forks source link

Adding the "X-RequestDigest" multiple times causes 403: Forbidden responses on on premise environments #1010

Closed nathan-swannet closed 2 months ago

nathan-swannet commented 2 months ago

We migrated our code base to use the latest Pnp.Framework (v1.15.0). Everything runs fine in our E2E tests with the cloud environments. However multiple E2E tests targeting our on-premise SharePoint Subscription environment fail with 403: Forbidden responses.

A small snippet can reproduce this issue:

var networkCredential = new NetworkCredential("username", securePwd, "domain");
var authManager = new AuthenticationManager();
using (var clientContext = authManager.GetOnPremisesContext("http://sphost", networkCredential))
{
     clientContext.Load(clientContext.Web, web => web.Title);
     await clientContext.ExecuteQueryAsync();
}

This snippet generates the following 5 HTTP request:

HTTP Request 1 > POST http://sphost/_vti_bin/sites.asmx HTTP/1.1 > Content-Type: text/xml > SOAPAction: http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation > X-RequestForceAuthentication: trueHost: sphost > Content-Length: 335 > Expect: 100-continue > Connection: Keep-Alive > ```xml > > ``` > HTTP/1.1 401 Unauthorized > Content-Type: text/plain; charset=utf-8 > Server: Microsoft-IIS/10.0 > SPRequestDuration: 5 > SPIisLatency: 3 > WWW-Authenticate: NTLM > X-Powered-By: ASP.NET > MicrosoftSharePointTeamServices: 16.0.0.15601 > X-Content-Type-Options: nosniff > X-MS-InvokeApp: 1; RequireReadOnly > Date: Mon, 15 Apr 2024 08:08:30 GMT > Content-Length: 16 > Proxy-Support: Session-Based-Authentication > 401 UNAUTHORIZED
HTTP Request 2 > POST http://sphost/_vti_bin/sites.asmx HTTP/1.1 > Content-Type: text/xml > SOAPAction: http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation > X-RequestForceAuthentication: true > Authorization: NTLM AuthString1 > Host: sphost > Content-Length: 0 > ```xml > ``` > HTTP/1.1 401 Unauthorized > Server: Microsoft-IIS/10.0 > WWW-Authenticate: NTLM AuthString2 > SPRequestGuid: 9c2f1fa1-2048-8098-0000-08265643fb7c > request-id: 9c2f1fa1-2048-8098-0000-08265643fb7c > X-FRAME-OPTIONS: SAMEORIGIN > SPRequestDuration: 1 > SPIisLatency: 0 > X-Powered-By: ASP.NET > MicrosoftSharePointTeamServices: 16.0.0.15601 > X-Content-Type-Options: nosniff > X-MS-InvokeApp: 1; RequireReadOnly > Date: Mon, 15 Apr 2024 08:08:30 GMT > Content-Length: 0 > Proxy-Support: Session-Based-Authentication
HTTP Request 3 > POST http://sphost/_vti_bin/sites.asmx HTTP/1.1 > Content-Type: text/xml > SOAPAction: http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation > X-RequestForceAuthentication: true > Authorization: NTLM AuthString3 > Host: sphost > Content-Length: 335 > Expect: 100-continue > ```xml > > ``` > HTTP/1.1 200 OK > Cache-Control: private, max-age=0 > Content-Type: text/xml; charset=utf-8 > Server: Microsoft-IIS/10.0 > X-SharePointHealthScore: 0 > X-AspNet-Version: 4.0.30319 > SPRequestGuid: 9c2f1fa1-1076-8098-0000-04372bbd172d > request-id: 9c2f1fa1-1076-8098-0000-04372bbd172d > X-FRAME-OPTIONS: SAMEORIGIN > SPRequestDuration: 17 > SPIisLatency: 0 > Persistent-Auth: true > X-Powered-By: ASP.NET > MicrosoftSharePointTeamServices: 16.0.0.15601 > X-Content-Type-Options: nosniff > X-MS-InvokeApp: 1; RequireReadOnly > Date: Mon, 15 Apr 2024 08:08:30 GMT > Content-Length: 845 > ```xml > GeneratedDigestString1,15 Apr 2024 08:08:30 -00001800http://sphost16.0.15601.2022614.0.0.0,15.0.0.0 > ```
HTTP Request 4 > POST http://fw-test-spsub/_vti_bin/sites.asmx HTTP/1.1 > X-RequestDigest: GeneratedDigestString1,15 Apr 2024 08:08:30 -0000 > X-FORMS_BASED_AUTH_ACCEPTED: f > Content-Type: text/xml > SOAPAction: http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation > X-RequestForceAuthentication: true > Host: sphost > Content-Length: 356 > Expect: 100-continue > Accept-Encoding: gzip, deflate > ```xml > > ``` > HTTP/1.1 200 OK > Cache-Control: private, max-age=0 > Content-Type: text/xml; charset=utf-8 > Vary: Accept-Encoding > Server: Microsoft-IIS/10.0 > X-SharePointHealthScore: 0 > X-AspNet-Version: 4.0.30319 > SPRequestGuid: 9c2f1fa1-e07b-8098-0000-0542b45c7a3d > request-id: 9c2f1fa1-e07b-8098-0000-0542b45c7a3d > X-FRAME-OPTIONS: SAMEORIGIN > SPRequestDuration: 9 > SPIisLatency: 0 > X-Powered-By: ASP.NET > MicrosoftSharePointTeamServices: 16.0.0.15601 > X-Content-Type-Options: nosniff > X-MS-InvokeApp: 1; RequireReadOnly > Date: Mon, 15 Apr 2024 08:08:30 GMT > Content-Length: 845 > ```xml > GeneratedDigestString1,15 Apr 2024 08:08:30 -00001800http://fw-test-spsub16.0.15601.2022614.0.0.0,15.0.0.0 > ```
HTTP Request 5 > POST http://fw-test-spsub/_vti_bin/client.svc/ProcessQuery HTTP/1.1 > X-RequestDigest: GeneratedDigestString1,15 Apr 2024 08:08:30 -0000,GeneratedDigestString1,15 Apr 2024 08:08:30 -0000 > X-FORMS_BASED_AUTH_ACCEPTED: f > Content-Type: text/xml > X-RequestForceAuthentication: true > Host: sphost > Content-Length: 606 > Expect: 100-continue > Accept-Encoding: gzip, deflate > ```xml > > ``` > HTTP/1.1 403 FORBIDDEN > Cache-Control: private > Content-Type: application/json; charset=utf-8 > Server: Microsoft-IIS/10.0 > X-SharePointHealthScore: 0 > X-AspNet-Version: 4.0.30319 > SPRequestGuid: 9c2f1fa1-b085-8098-0000-0d5d6752c4c6 > request-id: 9c2f1fa1-b085-8098-0000-0d5d6752c4c6 > X-RequestDigest: GeneratedDigestString2,15 Apr 2024 08:08:31 -0000 > X-FRAME-OPTIONS: SAMEORIGIN > X-Powered-By: ASP.NET > MicrosoftSharePointTeamServices: 16.0.0.15601 > X-Content-Type-Options: nosniff > X-MS-InvokeApp: 1; RequireReadOnly > Date: Mon, 15 Apr 2024 08:08:31 GMT > Content-Length: 460 > ```json > [{"SchemaVersion":"15.0.0.0","LibraryVersion":"16.0.15601.20220","ErrorInfo":{"ErrorMessage":"The security validation for this page is invalid. Click Back in your Web browser, refresh the page, and try your operation again.","ErrorValue":null,"TraceCorrelationId":"9c2f1fa1-b085-8098-0000-0d5d6752c4c6","ErrorCode":-2130575251,"ErrorTypeName":"System.Runtime.InteropServices.COMException"},"TraceCorrelationId":"9c2f1fa1-b085-8098-0000-0d5d6752c4c6"}] > ```

You can see that the GetUpdatedFormDigestInformation is invoked 2 times (Request 3 & 4) and both responses are added to the X-RequestDigest header of request 5 resulting in the faulty header

X-RequestDigest: GeneratedDigestString1,15 Apr 2024 08:08:30 -0000,GeneratedDigestString1,15 Apr 2024 08:08:30 -0000

This header value causes the 403: forbidden response.

If I alter the following code in the AuthenticationManager.cs

https://github.com/pnp/pnpframework/blob/2cd8204f230759d34f9e07c18535cd9648625151/src/lib/PnP.Framework/AuthenticationManager.cs#L1366

to

webRequestEventArgs.WebRequestExecutor.RequestHeaders["X-RequestDigest"] = (sender as ClientContext).GetOnPremisesRequestDigestAsync().GetAwaiter().GetResult(); 

Everything works.