search

SharePoint / sp-dev-docs

SharePoint & Viva Connections Developer Documentation
https://docs.microsoft.com/en-us/sharepoint/dev/
Creative Commons Attribution 4.0 International
1.24k stars 1.01k forks source link

Getting Error while try to fetch records using SharePoint Rest API. #7784

Open visualscrapy opened 2 years ago

visualscrapy commented 2 years ago

What type of issue is this?

Question

What SharePoint development model, framework, SDK or API is this about?

SharePoint REST API

Target SharePoint environment

SharePoint Online

What browser(s) / client(s) have you tested

Additional environment details

Issue description

In My site, I had 50,000+ records so I am trying to fetch records using SharePoint rest API call. Ex: https://tenent.sharepoint.com/sites/.../_api/lists/getbytitle('Documents')/files$skiptoken=Paged=TRUE&p_ID=39000&$top=1000 once it's reached to specific records it's throwing error. Column '_mpSPSiteId' does not exist.

ghost commented 2 years ago

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

visualscrapy commented 2 years ago

Fetch all records from the SharePoint site using SharePoint rest API.

visualscrapy commented 2 years ago

using rest api call.

hack-tom commented 2 years ago

I'm having the same issue - this is specifically an error occurring from following the __next variable in the paginated response, please see below and ignore the debugging print statements.

nextUrl = "http://" + tenant  + "/sites/" + siteName + "/_api/files"
reqCount = 0
while nextAvailable == True:
    l = requests.get(nextUrl, headers=headers)

    if l.text[:25] != """{"d":{"results":[{"__meta""":
        print("({}) Request Failed".format(reqCount), nextUrl.replace("----", "[TENANT]").replace("----", "[SITE]"))
        print(l.text)
    else:
        print("({}) Request Successful".format(reqCount), nextUrl.replace("----", "[TENANT]").replace("----", "[SITE]"))

    items = json.loads(l.text)
    for item in items['d']['results']:
        if item["__metadata"]['type'] == "MS.FileServices.File":
            files.append(item)

    if "__next" in items['d']:
        nextUrl = items['d']["__next"]
    else: 
        nextAvailable = False
    reqCount += 1

image

Precise error message is: 

{"error":{"code":"-2147024809, System.ArgumentException","message":{"lang":"en-US","value":"Column '_mpSPSiteId' does not exist. It may have been deleted by another user."}}}

As far as I am aware, no columns have been renamed or deleted - it is highly unlikely that anyone else has modified either. This error is reliably repeatable, occurring at the same point (12th page request). If this was genuinely a columns issue I would expect the first 12 page requests to fail too.

Worth noting that this issue did not occur on four previous sites that this code has ran against, I have 13 further SharePoint sites to process excluding this failing one and will update again if I find more examples of this issue.

UPDATE: this has occured on three further sites - I have confirmed that the issue is not new by verifying the same code still works on one of the original (successful) sites. The depth of pagination does not seem to be a factor, with the sites taking different (but consistent & repeatable) numbers of next page requests to fail.

jerodj-cdata commented 2 years ago

Our (CData Software) customers have recently started encountering this issue as well.

The problem is identical to what @hack-tom describes: several requests to the files endpoint are successful until one of the "next" links returned from SharePoint gives the "'_mpSPSiteId' does not exist" error.

In our case, we see this even with an explicit "$select=Id%2cUrl" on the request.

hack-tom commented 1 year ago

Apologies for the late return - I managed to get in touch with the product team and after a lengthy investigation they have found and resolved the issue on their side, which I believe has been pushed to all sharepoint online customers. In my own testing, the issue has been resolved.