search

nextcloud / integration_onedrive

🗔 Integration of Microsoft OneDrive into Nextcloud
GNU Affero General Public License v3.0
32 stars 7 forks source link

Only first 200 files imported from OneDrive/Pictures/Camera imports #3

Closed shr3k closed 3 years ago

shr3k commented 3 years ago

Hi, this plugin imports only first 200 files from large folders like Camera imports. When you want to import rest, one must delete first 200 from OneDrive.

julien-nc commented 3 years ago

Hi, thanks for the bug report.

Are you sure the import process had finished and the migration settings page was saying so?

Files are not imported all at once, it is done by successive background jobs of 500 MB max download each. It is possible the OneDrive API has some limitations and can't return more than 200 files per directory...but it's not mentioned in the documentation.

I'll try to reproduce the problem. To be continued.

shr3k commented 3 years ago

Yes, import was succesfully finished, only this one folder with more than 200 files get synced only up to first 200. As I wrote, I can delete first 200 files from Onedrive and with next import it imports second batch and so far so on.

julien-nc commented 3 years ago

Very weird. If the files are not there anymore, it should import the same ones, I mean in the same order so you should see the same ones appear.

julien-nc commented 3 years ago

v0.0.4 fixes this issue. I didn't know the API could return paginated results. It is actually buried deep in their horrible documentation and weirdly implemented :grin:.

I'm interested to know if everything runs fine on your side too.

shr3k commented 3 years ago

Generated 1000 files with random content, 757 MB, synced with OneDrive, imported into Nextcloud, all OK.

julien-nc commented 3 years ago

Yay! Thanks for the feedback.