search

boredazfcuk / docker-icloudpd

An Alpine Linux container for the iCloud Photos Downloader command line utility
1.57k stars 149 forks source link

auto_delete not deleting photos #552

Closed Klench closed 2 months ago

Klench commented 2 months ago

My understanding of the auto_delete variable is that when this is set to true, the files that have previously been downloaded using this application will be deleted if they have been deleted on the Apple device and thus moved to the "Recently Deleted" folder. This is however not functioning for me. Please see my configuration (email address removed) further down.

Note that the until_found variable has not been set as it was in this old issue.

Log:

2024-04-19 23:19:01 INFO     All photos have been downloaded
2024-04-19 23:19:01 INFO     Deleting any files found in 'Recently Deleted'...
2024-04-19 23:19:04 INFO     Synchronisation complete for user

icloudpd.conf:

agentid=
albums_with_dates=false
apple_id=
auth_china=false
authentication_type=MFA
auto_delete=true
bark_device_key=
bark_server=
content_source_url=
convert_heic_to_jpeg=false
debug_logging=false
delete_accompanying=false
delete_after_download=false
delete_notifications=true
dingtalk_token=
directory_permissions=777
discord_id=
discord_token=
download_notifications=true
download_path=
file_permissions=666
folder_structure=none
gotify_app_token=
gotify_https=
gotify_server_url=
group=group
group_id=1000
icloud_china=
iyuu_token=
jpeg_path=
jpeg_quality=90
libraries_with_dates=false
media_id_delete=
media_id_download=
media_id_expiration=
media_id_startup=
media_id_warning=
nextcloud_delete=false
nextcloud_password=
nextcloud_upload=false
nextcloud_url=
nextcloud_username=
notification_days=7
notification_type=
photo_album=
photo_library=
photo_size=original
prowl_api_key=
pushover_sound=
pushover_token=
pushover_user=
recent_only=
set_exif_datetime=false
single_pass=false
skip_album=
skip_check=false
skip_download=false
skip_library=
skip_live_photos=true
skip_videos=false
startup_notification=true
synchronisation_delay=0
synchronisation_interval=86400
synology_ignore_path=false
telegram_chat_id=
telegram_polling=true
telegram_server=
telegram_silent_file_notifications=
telegram_token=
touser=
trigger_nextlcoudcli_synchronisation=
until_found=
user=user
user_id=1000
webhook_https=false
webhook_id=
webhook_path=/api/webhook/
webhook_port=8123
webhook_server=
wecom_id=
wecom_proxy=
wecom_secret=
boredazfcuk commented 2 months ago

Log file is too small for it to be meaningful. It looks fine to me.

It will only process the 'Recently Deleted' directory if it downloads a new photo. If you'd deleted 20 photos, but not added any new ones, then this is what I'd expect to see.

Klench commented 2 months ago

Sorry for leaving out useful information. It seems to me that it does not process 'Recently Deleted' directory even though I added a new photo. Just to clarify: as I have not specified any folder structure, all photos on my iCloud library will be put in /home/user/iCloud and the photos that have been deleted and moved to the 'Recently Deleted' directory on iCloud and that also existed in /home/user/iCloud prior to the post deletion sync will be removed. Correct?

Please see a more extensive log:

2024-04-21 00:00:48 INFO     ***** boredazfcuk/icloudpd container for icloud_photo_downloader v1.0.733 started *****
2024-04-21 00:00:48 INFO     ***** For support, please go here: https://github.com/boredazfcuk/docker-icloudpd *****
2024-04-21 00:00:48 INFO     Alpine Linux 3.19.1
2024-04-21 00:00:48 INFO     Python version: 3.11.8
2024-04-21 00:00:48 INFO     Loading configuration from: /config/icloudpd.conf
2024-04-21 00:00:49 INFO     Apple ID: --------------------
2024-04-21 00:00:49 INFO     Authentication Type: MFA
2024-04-21 00:00:49 INFO     Cookie path: /config/--------------------
2024-04-21 00:00:49 INFO     Cookie expiry notification period: 7
2024-04-21 00:00:49 INFO     Download destination directory: /home/user/iCloud
2024-04-21 00:00:49 INFO     Folder structure: none
2024-04-21 00:00:49 INFO     Synchronisation interval: 86400
2024-04-21 00:00:49 INFO     Synchronisation delay (minutes): 0
2024-04-21 00:00:49 INFO     Set EXIF date/time: false
2024-04-21 00:00:49 INFO     Auto delete: true
2024-04-21 00:00:49 INFO     Delete after download: false
2024-04-21 00:00:49 INFO     Photo size: original
2024-04-21 00:00:49 INFO     Single pass mode: false
2024-04-21 00:00:49 INFO     Skip download check: false
2024-04-21 00:00:49 INFO     Skip live photos: true
2024-04-21 00:00:49 INFO     Number of most recently added photos to download: Download All Photos
2024-04-21 00:00:49 INFO     Downloading photos from: Download All Photos
2024-04-21 00:00:49 INFO     Stop downloading when prexisiting files count is: Download All Photos
2024-04-21 00:00:49 INFO     Skip videos: false
2024-04-21 00:00:49 INFO     Convert HEIC to JPEG: false
2024-04-21 00:00:49 INFO     Downloading from: icloud.com
2024-04-21 00:00:49 INFO     Authentication domain: com
2024-04-21 00:00:49 INFO     Ignore Synology extended attribute directories: Disabled
2024-04-21 00:00:51 INFO     Directory is writable: /config/python_keyring/
2024-04-21 00:00:51 INFO     Check download directory mounted correctly...
2024-04-21 00:00:51 INFO     Failsafe file /home/user/iCloud/.mounted exists, continuing
2024-04-21 00:00:51 INFO     Directory is writable: /config/python_keyring/
2024-04-21 00:00:51 INFO     Keyring file exists, continuing
2024-04-21 00:00:51 INFO     Sync user: user
2024-04-21 00:00:51 INFO     Synchronisation starting at 00:00:51
2024-04-21 00:00:51 INFO     Keyring file exists, continuing
2024-04-21 00:00:51 INFO     Check download directory mounted correctly...
2024-04-21 00:00:51 INFO     Failsafe file /home/user/iCloud/.mounted exists, continuing
2024-04-21 00:00:51 INFO     Check for new files using password stored in keyring file
2024-04-21 00:00:51 INFO     Generating list of files in iCloud. This may take a long time if you have a large photo collection. Please be patient. Nothing is being downloaded at this time
/home/user/iCloud/IMG_1966.HEIC

...

/home/user/iCloud/IMG_4386_HEVC.MOV
2024-04-15 23:36:14 INFO     Check successful
2024-04-15 23:36:14 INFO     New files detected: 2174
2024-04-15 23:36:16 DEBUG    Authenticating...
2024-04-15 23:36:22 DEBUG    Looking up all photos and videos from album All Photos...
2024-04-15 23:36:22 INFO     Downloading 3829 original photos and videos to /home/user/iCloud ...
2024-04-15 23:36:27 DEBUG    Downloading /home/user/iCloud/IMG_1948.HEIC...
2024-04-15 23:36:29 INFO     Downloaded /home/user/iCloud/IMG_1948.HEIC
2024-04-15 23:36:29 DEBUG    Downloading /home/user/iCloud/IMG_1947.HEIC...
2024-04-15 23:36:30 INFO     Downloaded /home/user/iCloud/IMG_1947.HEIC
2024-04-15 23:36:30 DEBUG    Downloading /home/user/iCloud/IMG_1946.HEIC...
2024-04-15 23:36:31 INFO     Downloaded /home/user/iCloud/IMG_1946.HEIC
2024-04-15 23:36:31 DEBUG    Downloading /home/user/iCloud/IMG_1945.HEIC...
2024-04-15 23:36:31 INFO     Downloaded /home/user/iCloud/IMG_1945.HEIC
2024-04-15 23:36:31 DEBUG    Downloading /home/user/iCloud/IMG_1944.HEIC...
2024-04-15 23:36:32 INFO     Downloaded /home/user/iCloud/IMG_1944.HEIC
2024-04-15 23:36:32 DEBUG    Downloading /home/user/iCloud/IMG_1942.HEIC...
2024-04-15 23:36:32 INFO     Downloaded /home/user/iCloud/IMG_1942.HEIC
2024-04-15 23:36:32 DEBUG    Downloading /home/user/iCloud/IMG_1941.HEIC...
2024-04-15 23:36:33 INFO     Downloaded /home/user/iCloud/IMG_1941.HEIC
2024-04-15 23:36:33 DEBUG    /home/user/iCloud/IMG_1938.HEIC already exists

...

2024-04-21 00:04:29 DEBUG    /home/user/iCloud/P1010001.JPG already exists
2024-04-21 00:04:29 INFO     All photos have been downloaded
2024-04-21 00:04:29 INFO     Deleting any files found in 'Recently Deleted'...
2024-04-21 00:04:32 INFO     New files downloaded: 1
2024-04-21 00:04:32 INFO     Synchronisation complete for user
2024-04-21 00:04:32 INFO     Web cookie expires: 2024-05-31 @ 07:01:17
2024-04-21 00:04:32 INFO     Multifactor authentication cookie expires: 2024-05-20 @ 21:17:09
2024-04-21 00:04:32 INFO     Days remaining until expiration: 29
2024-04-21 00:04:32 INFO     Synchronisation ended at 00:04:32
2024-04-21 00:04:32 INFO     Total time taken: 00:03:41
2024-04-21 00:04:32 INFO     Next synchronisation at 00:00:51
boredazfcuk commented 2 months ago

So you have 3k files in your photo stream, but 2k of them are new? That suggests only 1k of them were downloaded on the first run, or you took 2k photos as a test. That's a bit odd.

I'd start by adding a folder structure. Year as a bare minimum, ideally Year & Month. Setting a folder increases the number of photos it can sync. Setting the folder structure to none caps you at 10k photos. Having just Year allows 10k per year. Having month allows 10k per month.

Klench commented 2 months ago

I don't know why it did download only a subset of the photos. However, it works after I created a folder structure! I chose year and month only to keep the structure neat and I will not have that many photos so it will be fine.

Thank you for the support and thank you for this awesome docker container! I can now use it in conjunction with Immich 🙂

2024-04-22 22:03:42 INFO     All photos have been downloaded
2024-04-22 22:03:42 INFO     Deleting any files found in 'Recently Deleted'...
2024-04-22 22:03:44 DEBUG    Deleting /home/user/iCloud/2024/03/IMG_1807.PNG...
2024-04-22 22:03:44 INFO     Deleted /home/user/iCloud/2024/03/IMG_1807.PNG
2024-04-22 22:03:44 DEBUG    Deleting /home/user/iCloud/2024/01/IMG_1369.PNG...
2024-04-22 22:03:44 INFO     Deleted /home/user/iCloud/2024/01/IMG_1369.PNG
2024-04-22 22:03:45 INFO     New files downloaded: 1
2024-04-22 22:03:45 INFO     Number of files deleted: 2
2024-04-22 22:03:46 INFO     Synchronisation complete for user