search

mandarons / icloud-docker

Dockerized iCloud Client - make a local copy of your iCloud documents and photos, and keep it automatically up-to-date.
BSD 3-Clause "New" or "Revised" License
1.15k stars 49 forks source link

[BUG] Broken sync of packages #174

Closed tymmej closed 11 months ago

tymmej commented 11 months ago

Describe the bug https://github.com/mandarons/icloud-drive-docker/commit/f85aa35f6573d2aa2a9ab9df480489ec403699e7 seems to break syncing packages.

To Reproduce Try to sync package with remove_obsolete: true

Expected behavior Packages sync successfully, observed behavior: empty folder.

Logs

2023-11-14T17:34:42.235510294Z 2023-11-14 17:34:42,235 :: INFO :: root :: sync_drive.py :: 177 :: Successfully unpacked the package /app/icloud/drive/Documents/PhotoStatistica.photostats.zip.
2023-11-14T17:34:42.233163379Z 2023-11-14 17:34:42,232 :: INFO :: root :: sync_drive.py :: 156 :: Unpacking /app/icloud/drive/Documents/PhotoStatistica.photostats.zip to /app/icloud/drive/Documents
2023-11-14T17:34:40.097011171Z 2023-11-14 17:34:40,096 :: INFO :: root :: sync_drive.py :: 193 :: Downloading /app/icloud/drive/Documents/PhotoStatistica.photostats ...
2023-11-14T17:34:40.096310648Z 2023-11-14 17:34:40,095 :: INFO :: root :: sync_drive.py :: 113 :: Changes detected: local_modified_time is 1699915738, remote_modified_time is 1679320573, local_package_size is 0 and remote_package_size is 4070.
...
2023-11-14T17:50:04.497817862Z 2023-11-14 17:50:04,497 :: INFO :: root :: sync_drive.py :: 238 :: Removing /app/icloud/drive/Documents/PhotoStatistica.photostats/photoSources ...
2023-11-14T17:50:04.497385378Z 2023-11-14 17:50:04,497 :: INFO :: root :: sync_drive.py :: 238 :: Removing /app/icloud/drive/Documents/PhotoStatistica.photostats/charts ...
2023-11-14T17:50:04.496974661Z 2023-11-14 17:50:04,496 :: INFO :: root :: sync_drive.py :: 238 :: Removing /app/icloud/drive/Documents/PhotoStatistica.photostats/viewSettings ...

On filesystem is empty folder /app/icloud/drive/Documents/PhotoStatistica.photostats.

Configuration remove_obsolete: true for drive.

tymmej commented 11 months ago

That code is necessary to add files from package to list with "known" files to prevent deletion when remove_obsolete is set to true. process_package nor download_files does not do it. process_file add to list only "base" name (name of package, not its content).

mandarons commented 11 months ago

That's interesting. For packages, only package directory path should be enough to retain in the files list. remove_obsolete should have removed the local package directory and its contents. But apparently, it isn't removing it.

Looking into it....

tymmej commented 11 months ago

Removing files are naive: https://github.com/mandarons/icloud-drive-docker/blob/ac2a960d29790504c63f0391a4bc622fade083cb/src/sync_drive.py#L235

rglob finds all files, but on the list of wanted files there's only package directory path. And only package directory path is left after running remove_obsolete ().

Another solution is to change condition in https://github.com/mandarons/icloud-drive-docker/blob/ac2a960d29790504c63f0391a4bc622fade083cb/src/sync_drive.py#L237 to check if there's any part of path on list of wanted files.

tymmej commented 11 months ago

https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.is_relative_to looks like what can be used but: a) is deprecated b) complexity might be a problem because it should be called for all entries in files for each found file.