The attempted operation is prohibited because it exceeds the list view threshold

[azamsolix]

Hi sir, There are more than 10k files in one folder and i am trying to retrieve all of them but i am getting error "The attempted operation is prohibited because it exceeds the list view threshold".However,I am write a query query({ Expand: ["Folders,Files,Files/Author"],Top:6000,GetAllItems:true }) and this is url it is generating https://sitesites.sharepoint.com/sites/vvv/_api/web/getFolderByServerRelativeUrl('/sites/vvv/Shared Documents/nnn/10k-50MB')?$top=6000&$expand=Folders,Files,Files/Author. According to my understanding it should return 6000 files from the folder instead it is returning error. Below is the code which i wrote.Please, help me with this.

var spauth = require('node-sp-auth'); var $REST = require("gd-sprest"); var { List, SPTypes, Web } = require("gd-sprest"); var fetch = require('node-fetch'); // Connect to SPO var url = "https://site.sharepoint.com"; spauth.getAuth(url, { username: "username", password: "====", domain:'https://site.sharepoint.com', online: true }).then(async(options) => { // Log console.log("Connected to SPO", options); try { let path = ""; path = "/sites/10k-50MB"; // Code Continues in 'Generate the Request' // Get the web var web = $REST.Web(url); if (path != "") var info = web.getFolderByServerRelativeUrl(path).query({ Expand: ["Folders,Files,Files/Author"],Top:2,GetAllItems:true }).getInfo(); else { var info = web.Lists("Documents").RootFolder().getInfo(); } // Copy the headers from the SP authentication var authobj = {}; for (var key in options.headers) { // Set the header info.headers[key] = options.headers[key]; } authobj.headers = info.headers; fetch(info.url, authobj).then(async(res)=> { let ans = await res.json(); console.log('ans', ans); } ) } catch (err) { console.log("error after connected", err); }; }).catch((er) => { console.log("Connecting to SPO failed", er); });

Thanks, Azam

[gunjandatta]

You can only get a max of 5k results, per this article. Try doing a Top: 5000 and see if you can get the results. The "GetAllItems" flag will automate the check for "next" results and execute it until all items are retrieved. If you are still getting an error, then try simpler queries (remove the Expand option) and see if you can get the results.

The "GetAllItems" flag is only for use when executing against the REST API directly through JavaScript in the browser. The NodeJS option will require you to look at the "next" paging option and get the next set of results, but per your code example it doesn't look like you need that part. Please try to execute the request in the browser to ensure it's valid first.

[azamsolix]

Hi sir, Apologies for the late reply. Note:we are not calling rest api from browser. we are calling it from backend as it shows in the sample code i shared before. As our requirement is to retrieve all files from folders and migrate them to our application.

I have tested by removing expand option and it is not returning Files Array in response instead of the it is returning URL to retrieve Files and calling upon that url i am getting same threshold error and i have also tested with different folder and query method is not working.

i.e ` //In this path,10Files folder is having 10 files and i want to retrieve top 5 files path = "/sites/vv/Shared Documents/vvv/10Files";

    var web = $REST.Web(url);

//Here i am using Top to get top 5 files // var info = web.getFolderByServerRelativeUrl(dir).query({ Expand: ["Folders,Files,Files/Author"],Top:5,GetAllItems:true }).getInfo();

//generated url https://sitesites.sharepoint.com/sites/vv/_api/web/getFolderByServerRelativeUrl('/sites/vv/Shared Documents/vvv/10Files')?$top=5&$expand=Folders,Files,Files/Author

var authobj = {}; for (var key in options.headers) { // Set the header info.headers[key] = options.headers[key]; } authobj.headers = info.headers;

//however,Instead of returning top 5 files it is returning all 10 files, fetch(,info.url,authobj).then((async(res)=>{ res=await res.json() console.log('res',res) })) Is there something i am doing wrong.Please help me with this.

Thank you Azam

[gunjandatta]

If you can't execute the request in the browser, then it won't work in code. This is the best way to test your query out. My best guess is that it's not working due to the size of the library. You are going to have to test REST API queries in the browser with a simple request then slowly add the expand/select/top options.

Does that make sense?

[gunjandatta]

Closing this issue out. Please reopen if you still need help with this.