search

mandarons / icloud-drive-docker

Dockerized iCloud Client - make a local copy of your iCloud documents and photos, and keep it automatically up-to-date.
BSD 3-Clause "New" or "Revised" License
968 stars 46 forks source link

[FEATURE] Avoid redownloading unchanged files #187

Closed briansunter closed 4 months ago

briansunter commented 5 months ago

Use case As a user continually backing up their icloud drive documents each day/week, I want to avoid redownloading unchanged files so that I can sync more quickly and save on network traffic.

Describe the solution you'd like Possibly checking an updatedAt field in the icloud api (if it exists), and not redownloading the file if it hasn't changed since the last sync.

Describe alternatives you've considered Multiple configurations each syncing an individual folder, so the syncs complete more quickly.

Additional context Really appreciate you making this plugin. Just used it to download 500+gb over ~24 hours. When I run it a second time, it seems to be redownloading each file. Would be great if there was an optimization to make it run more quickly on subsequent runs. Ideally I could do daily backups and it would only download the files that changed since the last backup.

mandarons commented 4 months ago

It shouldn't re-download the files again if they are not changed (no changes in file size and no changes in last updated timestamp).

https://github.com/mandarons/icloud-drive-docker/blob/6fc91281954620edb2d103c97f72317d235e7293/src/sync_drive.py#L131-L135

Please share your logs if you still believe that files are being redownloaded without any changes to them.

briansunter commented 4 months ago

I ran it again and it stopped happening. Not sure what happened. Thank you for confirming the expected behavior and thanks for the awesome app. Just completed 750gb sync and seems to only be pulling the new files now!