jasonrollins
commented
3 years ago This is a limitation of your SharePoint server, that is put in by default by Microsoft. You won't be able to get around it, unless you are the administrator of your SharePoint site and can increase this limit.
On Mon, Jun 28, 2021 at 4:33 AM sme31 @.***> wrote:
I am (a novice) using shareplum to upload 35,000 documents to a single document library, and when I get to 5,000 documents I'm getting a500 Server Error.
The process I am using is to upload the file, and then set the metadata using this module. (Not my code, I've taken it from here https://multiportmedia.com/uploading-files-to-sharepoint-using-python-part-1/ )
def addMeta(self): instanceOfDocumentLibrary = self.site.List(self.sharePointFolder) fields = ['Name', 'ID'] query = {'Where': [('Contains', 'Name', self.fileName)]} GetData = instanceOfDocumentLibrary.GetListItems(fields=fields, query=query) getDataDict = GetData[-1] util = Util() payload = util.mergeDict(getDataDict, self.customeMetaDict) update_data = [payload] instanceOfDocumentLibrary.UpdateListItems(data=update_data,kind='Update')
The error is generated when the GetData line is executed. The file gets uploaded fine, but fails in this module. I suspect what is happening is that when the List is instantiated (instanceOfDocumentLibrary) it only contains 5,000 documents (as per the SharePoint 5,000 document view limit), and so when we do the GetListItems it can't find the 5,001th document that we've just uploaded because instanceOfDocumentLibrary only contains 5,000 documents.
Dipping into the Shareplum code (list.py) I see row_limit=0, which in the comments says the row limit should be unlimited, but I suspect it isn't unlimited. Could it be that leaving row_limit=0, it is defaulting to 5,000 (i.e. the SharePoint view limit)? If we make row_limit=50,000 (for example) would that solve the issue?
Way out of my depth here, so any suggestions more than welcome.
Thanks
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jasonrollins/shareplum/issues/167, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOVTVUZF4MWPJCSTUKJIODTVA6XVANCNFSM47NSUALA .
sme31
oederp1
I am (a novice) using shareplum to upload 35,000 documents to a single document library, and when I get to 5,000 documents I'm getting a 500 Server Error.
The process I am using is to upload the file, and then set the metadata using this module. (Not my code, I've taken it from here https://multiportmedia.com/uploading-files-to-sharepoint-using-python-part-1/)
def addMeta(self): instanceOfDocumentLibrary = self.site.List(self.sharePointFolder) fields = ['Name', 'ID'] query = {'Where': [('Contains', 'Name', self.fileName)]} GetData = instanceOfDocumentLibrary.GetListItems(fields=fields, query=query) getDataDict = GetData[-1] util = Util() payload = util.mergeDict(getDataDict, self.customeMetaDict) update_data = [payload] instanceOfDocumentLibrary.UpdateListItems(data=update_data,kind='Update')
The error is generated when the GetData line is executed. The file gets uploaded fine (by a different module), but fails in this module. I suspect what is happening is that when the List is instantiated (instanceOfDocumentLibrary) it only contains 5,000 documents (as per the SharePoint 5,000 document view limit), and so when we do the GetListItems it can't find the 5,001th document that we've just uploaded because instanceOfDocumentLibrary only contains 5,000 documents.
Dipping into the Shareplum code (list.py) I see row_limit=0, which in the comments says the row limit should be unlimited, but I suspect it isn't unlimited. Could it be that leaving row_limit=0, it is defaulting to 5,000 (i.e. the SharePoint view limit)? If we make row_limit=50,000 (for example) would that solve the issue?
Way out of my depth here, so any suggestions more than welcome.
Thanks