'Access denied' from Microsoft.SharePoint.Client ExecuteQueryRetry - previously working queries are now coming back with error

brett-src commented 1 week ago

Target SharePoint environment

SharePoint Online

What SharePoint development model, framework, SDK or API is this about?

SharePoint CSOM

Developer environment

Windows

What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[ ] 💥 Microsoft Edge
[ ] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[X] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details

No response

Describe the bug / error

SharePoint CSOM scripts that are running in an Azure WebJob have been stable for a long time (years). Recently (about a week ago), these stable scripts started reporting some 'Access denied' failures. These failures occur upon using ExecuteQueryRetry from the ClientContext.

Keep in mind that:

The service account that runs these scripts has up-to-date and valid credentials (it should not be denied in any of its actions)
Many of the steps in the script work, but certain ones are only recently failing

Two operations that I've seen fail so far:

Add to a FileCollection using FileCreationInformation. In this step, we attempt to overwrite the master page in the master page catalog.
DeleteObject on a File. In this step, we attempt to delete a page from Site Pages.

I imagine that some downstream operations might fail but I cannot progress far enough into my scripts to determine what will work.

Last successful run: 6/18/2024 10:01:17 AM (US Central Time) First failure run: 6/20/2024 07:03:30 PM (US Central Time)

My suspicion is that some update happened within SharePoint which perhaps changed the way certain CSOM operations work. Is there any idea as to what could be going on?

Steps to reproduce

Below are the two scripts referenced in the error.

Overwrite master page

var web = clientContext.Site.OpenWeb("relative_site_url_here");
var masterPageCatalog = web.GetCatalog((int)ListTemplateType.MasterPageCatalog);
clientContext.Load(masterPageCatalog.RootFolder);
clientContext.ExecuteQueryRetry();

var masterPageBytes = MethodThatGetsBytesOfAnotherMasterPage();
var masterPageString = Encoding.UTF8.GetString(masterPageBytes);

var masterPageFileCreationInfo = new FileCreationInformation()
{
  Url = string.Format(@"{0}/{1}", masterPageCatalog.RootFolder.ServerRelativeUrl, "seattle.master"),
  Overwrite = true,
  Content = Encoding.UTF8.GetBytes(masterPageString)
};

var updatedMasterPage = masterPageCatalog.RootFolder.Files.Add(masterPageFileCreationInfo);
clientContext.Load(updatedMasterPage);
// Failure of 'Access denied' occurs here
clientContext.ExecuteQueryRetry();

Delete a page from Site Pages

var web = clientContext.Site.OpenWeb("relative_site_url_here");
var sitePages = web.Lists.GetByTitle("Site Pages");
clientContext.Load(sitePages.RootFolder);
clientContext.ExecuteQueryRetry();

// Delete 'How To Use This Library' file if it is found
var howToUsePage = default(Microsoft.SharePoint.Client.File);
try
{
  howToUsePage = sitePages.RootFolder.GetFile("How To Use This Library.aspx");
}
catch { howToUsePage = null; }
if (howToUsePage != null)
{
  howToUsePage.DeleteObject();
  // Failure of 'Access denied' occurs here
  clientContext.ExecuteQueryRetry();
}

Expected behavior

The stable CSOM scripts should not have any Access denied errors.

brett-src commented 6 days ago

Hi there - I am hoping I can get some support! My client's automated processes are broken.

Tagging @qianghuang1 as I've worked with you previously on an issue. Can you help me out? Or let me know who to contact?

Tran-Kien commented 5 days ago

Hi, we are also experiencing the same problem. We're using CSOM to create a sub site, set its masterpage, break permission inheritance. Executed with an account that is Site Col Admin.

It was working last week, now it is giving the following Access is denied response.

[ { "SchemaVersion": "15.0.0.0", "LibraryVersion": "16.0.25019.12006", "ErrorInfo": { "ErrorMessage": "Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))", "ErrorValue": null, "TraceCorrelationId": "[guid]", "ErrorCode": -2147024891, "ErrorTypeName": "System.UnauthorizedAccessException" }, "TraceCorrelationId": "[guid]" } ]

Here's the code snippet.

            var webCreateInfo = new SP.WebCreationInformation();

            webCreateInfo.set_title(siteTitle);
            webCreateInfo.set_language(templateLanguage);
            webCreateInfo.set_url(genUrl);
            webCreateInfo.set_useSamePermissionsAsParentSite(useSamePermissionsAsParentSite);
            webCreateInfo.set_webTemplate(customSiteTemplateId);

            // Add sub site using newWeb global object
            newWeb = web.get_webs().add(webCreateInfo);

            //set masterpage
            newWeb.set_masterUrl(masterpageUrl);

            // Set nav
            newWeb.get_navigation().set_useShared(useShared);

            // BREAK INHERITING GROUPS
            if( useSamePermissionsAsParentSite == "false")
            {
                newWeb.breakRoleInheritance(copyRoleAssignments, clearSubscopes);
            }

            // Disable members sharing if (newWeb.get_membersCanShare())
            newWeb.set_membersCanShare(false);

            if (accessRequestEmail != null)
            {
                newWeb.set_requestAccessEmail(accessRequestEmail);
            }

            newWeb.update();

            context.executeQueryAsync()

brett-src commented 5 days ago

@Tran-Kien sorry to hear, but it's reassuring to know that we're not alone. Is seems more than likely that some SharePoint update caused this type of behavior. Please note from my original post the timeframe in which we saw the last success vs the first failure.

Last successful run: 6/18/2024 10:01:17 AM (US Central Time) First failure run: 6/20/2024 07:03:30 PM (US Central Time)

Do you have any knowledge of your last success or first failure that can help narrow this window?

@qianghuang1 again tagging you in the hopes that someone can start triaging this breaking issue.

qianghuang1 commented 3 days ago

@brett-src Thanks for your patience. The better way in my mind to get quick support is to open a ticket by the tenant that run into the issue via Microsoft Support System instead of asking in GitHub. For the issue itself, i didn't own the service, but looks like it lacks permission to do the sensitive operations. I would suggest open a ticket, list the permission that your application identity had (used by the azure webjob), and provide a correlation id of the failed csom request so that supports/devs can better investigate on the issue. Meanwhile, you can try to grant the app with higher privilege to see if it can mitigate this issue (like files.readwrite.all & sites.readwrite.all). (BTW, it's holiday season in US due to the Independence Day, so the response in github may be slower)

SharePoint / sp-dev-docs