boredazfcuk / docker-icloudpd

An Alpine Linux container for the iCloud Photos Downloader command line utility
1.58k stars 149 forks source link

/config/icloudpd.conf: line 22: {:%Y/%m/%d}: not found #488

Closed rustig136 closed 5 months ago

rustig136 commented 5 months ago

Hi, it must be something simple but I can't figure out why /config/icloudpd.conf can't be found. The application updates/stores the running config in the 'config' location '/home/pi/appdata/etc/icloudpd/icloudpd_config' and everything seems to be running fine.

Can you help me remove this error, please?

Thanks! M

pi@docker:~/appdata/etc/icloudpd $ docker-compose logs Attaching to icloudpd icloudpd | icloudpd | 2024-01-14 19:13:31 INFO boredazfcuk/icloudpd container for icloud_photo_downloader v1.0.714 started icloudpd | 2024-01-14 19:13:31 INFO For support, please go here: https://github.com/boredazfcuk/docker-icloudpd icloudpd | 2024-01-14 19:13:31 INFO Alpine Linux 3.19.0 icloudpd | 2024-01-14 19:13:31 INFO Python version: 3.11.6 icloudpd | 2024-01-14 19:13:31 INFO Loading configuration from: /config/icloudpd.conf

icloudpd | /usr/local/bin/sync-icloud.sh: /config/icloudpd.conf: line 22: {:%Y/%m/%d}: not found

icloudpd | 2024-01-14 19:13:31 INFO Apple ID: xxx icloudpd | 2024-01-14 19:13:31 INFO Authentication Type: MFA icloudpd | 2024-01-14 19:13:31 INFO Cookie path: /config/xxx icloudpd | 2024-01-14 19:13:31 INFO Cookie expiry notification period: 7 icloudpd | 2024-01-14 19:13:31 INFO Download destination directory: /home/xxx/iCloud/ icloudpd | 2024-01-14 19:13:31 INFO Folder structure: {:%Y/%m/%d} icloudpd | 2024-01-14 19:13:31 INFO Synchronisation interval: 43200 icloudpd | 2024-01-14 19:13:31 INFO Synchronisation delay (minutes): 0 icloudpd | 2024-01-14 19:13:31 INFO Set EXIF date/time: false icloudpd | 2024-01-14 19:13:31 INFO Auto delete: false icloudpd | 2024-01-14 19:13:31 INFO Delete after download: false icloudpd | 2024-01-14 19:13:31 INFO Photo size: original icloudpd | 2024-01-14 19:13:31 INFO Single pass mode: false icloudpd | 2024-01-14 19:13:31 INFO Skip download check: false icloudpd | 2024-01-14 19:13:31 INFO Skip live photos: false icloudpd | 2024-01-14 19:13:31 INFO Number of most recently added photos to download: Download All Photos icloudpd | 2024-01-14 19:13:31 INFO Downloading photos from: Download All Photos icloudpd | 2024-01-14 19:13:31 INFO Stop downloading when prexisiting files count is: Download All Photos icloudpd | 2024-01-14 19:13:31 INFO Live photo size: original icloudpd | 2024-01-14 19:13:31 INFO Skip videos: false icloudpd | 2024-01-14 19:13:31 INFO Convert HEIC to JPEG: true icloudpd | 2024-01-14 19:13:31 INFO Pushover notifications enabled icloudpd | 2024-01-14 19:13:31 INFO Pushover user: xxx icloudpd | 2024-01-14 19:13:31 INFO Pushover token: xxx icloudpd | 2024-01-14 19:13:31 INFO Sending Pushover startup notification icloudpd | 2024-01-14 19:13:32 INFO Downloading from: icloud.com icloudpd | 2024-01-14 19:13:32 INFO Authentication domain: com icloudpd | 2024-01-14 19:13:32 INFO Ignore Synology extended attribute directories: Disabled icloudpd | 2024-01-14 19:13:34 INFO Directory is writable: /config/python_keyring/ icloudpd | 2024-01-14 19:13:34 INFO Check download directory mounted correctly... icloudpd | 2024-01-14 19:13:34 INFO Failsafe file /home/xxx/iCloud//.mounted exists, continuing icloudpd | 2024-01-14 19:13:34 INFO Directory is writable: /config/python_keyring/ icloudpd | 2024-01-14 19:13:34 INFO Keyring file exists, continuing icloudpd | 2024-01-14 19:13:34 INFO Sync user: xxx icloudpd | 2024-01-14 19:13:34 INFO Synchronisation starting at 19:13:34 icloudpd | 2024-01-14 19:13:34 INFO Keyring file exists, continuing icloudpd | 2024-01-14 19:13:34 INFO Check download directory mounted correctly... icloudpd | 2024-01-14 19:13:34 INFO Failsafe file /home/xxx/iCloud//.mounted exists, continuing icloudpd | 2024-01-14 19:13:34 INFO Check for new files using password stored in keyring file icloudpd | 2024-01-14 19:13:34 INFO Generating list of files in iCloud. This may take a long time if you have a large photo collection. Please be patient. Nothing is being downloaded at this time


_Docker-compose.yml_

version: "3.8"

volumes:
   config:
     name: icloudpd_config

services:
   icloudpd:
      container_name: icloudpd
      hostname: icloudpd_xxx
      environment:
         - apple_id=xxx
         - user=xxx
         - user_id=1000
         - group=group
         - group_id=1000
         - TZ=Europe/Amsterdam
         - download_path=/home/xxx/iCloud/
         - synchronisation_interval=43200
         - synchronisation_delay=0
         - notification_days=7
         - authentication_type=MFA
         - directory_permissions=750
         - file_permissions=640
         - folder_structure= {:%Y/%m/%d}
         - skip_check=false
         - download_notifications=true
         - delete_notifications=true
         - startup_notification=true
         - convert_heic_to_jpeg=true
         - notification_type=Pushover
         - notification_title=iCloudPD
         - pushover_user=xxx
         - pushover_token=xxx

      image: boredazfcuk/icloudpd
      healthcheck:
         test: /usr/local/bin/healthcheck.sh
         start_period: 30s
      restart: always
      volumes:
#       - config:/home/pi/appdata/etc/icloudpd/icloudpd_config          # dit not help :(
         - /home/pi/appdata/etc/icloudpd/icloudpd_config:/config
         - /mnt/backup/icloud:/home/xxx/
boredazfcuk commented 5 months ago
      environment:
         - apple_id=xxx
         - user=xxx
         - user_id=1000
         - group=group
         - group_id=1000
         - TZ=Europe/Amsterdam
         - download_path=/home/xxx/iCloud/
         - synchronisation_interval=43200
         - synchronisation_delay=0
         - notification_days=7
         - authentication_type=MFA
         - directory_permissions=750
         - file_permissions=640
         - folder_structure= {:%Y/%m/%d}
         - skip_check=false
         - download_notifications=true
         - delete_notifications=true
         - startup_notification=true
         - convert_heic_to_jpeg=true
         - notification_type=Pushover
         - notification_title=iCloudPD
         - pushover_user=xxx
         - pushover_token=xxx

Get rid of all that, then check that your /config/icloudpd.conf file has the line:

folder_structure={:%Y/%m/%d}
rustig136 commented 5 months ago

will give it a go, thanks for your feedback!

rustig136 commented 5 months ago

same error raised.

My initial test was without "photo_library=" and it was downloading it using the {:%Y/%m/%d} folder structure but still was throwing the error. Later I noticed that the shared library wasn't downloaded so I have added the photo_library= parameter. It only downloads the PrimarySync and noting more. It also doesn't use the {:%Y/%m/%d} folder structure.

Thank you, M

2024-01-21 20:58:10 INFO boredazfcuk/icloudpd container for icloud_photo_downloader v1.0.714 started 2024-01-21 20:58:10 INFO For support, please go here: https://github.com/boredazfcuk/docker-icloudpd 2024-01-21 20:58:10 INFO Alpine Linux 3.19.0 2024-01-21 20:58:10 INFO Python version: 3.11.6 2024-01-21 20:58:10 INFO Loading configuration from: /config/icloudpd.conf /usr/local/bin/sync-icloud.sh: /config/icloudpd.conf: line 22: {:%Y/%m/%d}: not found 2024-01-21 20:58:18 INFO Apple ID: xxx 2024-01-21 20:58:18 INFO Authentication Type: MFA 2024-01-21 20:58:18 INFO Cookie path: /config/xxx 2024-01-21 20:58:18 INFO Cookie expiry notification period: 7 2024-01-21 20:58:18 INFO Download destination directory: /home/xxx/iCloud/ 2024-01-21 20:58:18 INFO Folder structure: {:%Y/%m/%d} 2024-01-21 20:58:18 INFO Synchronisation interval: 86400 2024-01-21 20:58:18 INFO Synchronisation delay (minutes): 0 2024-01-21 20:58:18 INFO Set EXIF date/time: false 2024-01-21 20:58:18 INFO Auto delete: false 2024-01-21 20:58:18 INFO Delete after download: false 2024-01-21 20:58:18 INFO Photo size: original 2024-01-21 20:58:18 INFO Single pass mode: false 2024-01-21 20:58:18 INFO Skip download check: false 2024-01-21 20:58:18 INFO Skip live photos: false 2024-01-21 20:58:18 INFO Number of most recently added photos to download: Download All Photos 2024-01-21 20:58:18 INFO Downloading photos from library: PrimarySync,SharedSync-XXXXXXX-XXXX-XXXX-BF62-AA0XXX13E0210 2024-01-21 20:58:18 INFO Stop downloading when prexisiting files count is: Download All Photos 2024-01-21 20:58:18 INFO Live photo size: original 2024-01-21 20:58:18 INFO Skip videos: false 2024-01-21 20:58:19 INFO Downloading from: icloud.com 2024-01-21 20:58:19 INFO Authentication domain: com 2024-01-21 20:58:19 INFO Ignore Synology extended attribute directories: Disabled 2024-01-21 20:58:21 INFO Directory is writable: /config/python_keyring/ 2024-01-21 20:58:21 INFO Check download directory mounted correctly... 2024-01-21 20:58:21 INFO Failsafe file /home/xxxx/iCloud//.mounted exists, continuing 2024-01-21 20:58:21 INFO Directory is writable: /config/python_keyring/ 2024-01-21 20:58:21 INFO Keyring file exists, continuing 2024-01-21 20:58:21 INFO Sync user: user 2024-01-21 20:58:21 INFO Synchronisation starting at 20:58:21 2024-01-21 20:58:21 INFO Keyring file exists, continuing 2024-01-21 20:58:21 INFO Check download directory mounted correctly... 2024-01-21 20:58:21 INFO Failsafe file /home/xxx/iCloud//.mounted exists, continuing 2024-01-21 20:58:21 INFO Check for new files using password stored in keyring file 2024-01-21 20:58:21 INFO Generating list of files in iCloud. This may take a long time if you have a large photo collection. Please be patient. Nothing is being downloaded at this time

icloudpd.conf agentid= albums_with_dates=false apple_id=xxx auth_china=false authentication_type=MFA auto_delete=false bark_device_key= bark_server= content_source_url= convert_heic_to_jpeg=true debug_logging=false delete_accompanying=false delete_after_download=false delete_notifications=true dingtalk_token= directory_permissions=750 discord_id= discord_token= download_notifications=true download_path=/home/xxx/iCloud/ file_permissions=640 folder_structure= {:%Y/%m/%d} gotify_app_token= group=group group_id=1000 icloud_china= iyuu_token= jpeg_path= jpeg_quality=90 libraries_with_dates=false media_id_delete= media_id_download= media_id_expiration= media_id_startup= media_id_warning= nextcloud_delete=false nextcloud_password= nextcloud_upload=false nextcloud_url= nextcloud_username= notification_days=7 photo_album= photo_library="PrimarySync,SharedSync-REMOVED" photo_size=original prowl_api_key= recent_only= set_exif_datetime=false single_pass=false skip_album= skip_check=false skip_download=false skip_library= skip_live_photos=false skip_videos=false startup_notification=true synchronisation_delay=0 synchronisation_interval=43200 synology_ignore_path=false telegram_chat_id= telegram_polling=true telegram_server= telegram_silent_file_notifications= telegram_token= touser= trigger_nextlcoudcli_synchronisation= until_found= user_id=1000 webhook_https=false webhook_id= webhook_path=/api/webhook/ webhook_port=8123 webhook_server= wecom_id= wecom_proxy= wecom_secret=

boredazfcuk commented 5 months ago

Pretty sure it's still this bit you need:

folder_structure={:%Y/%m/%d}

your config file has:

folder_structure= {:%Y/%m/%d}
rustig136 commented 5 months ago

Yes and I have noticed that as well. When I added it without the space, like you suggested - it was throwing the same error and icloudpd changed the icloudpd.conf file to "folder_structure= {:%Y/%m/%d}" which I found strange.

If I removed icloudpd.conf and start the application it also gets created with the additional space. I'm going to change it in the docker-compose.yml file today to something else and see if it also gets copied into icloudpd.conf with space.

blackfox33 commented 5 months ago

Here is my config, maybe it helps. I ran into some issues as well and set up this docker container freshly a few weeks ago. It now runs fine.

agentid= albums_with_dates=false apple_id=##### auth_china=false authentication_type=MFA auto_delete=false bark_device_key= bark_server= content_source_url= convert_heic_to_jpeg=false debug_logging=false delete_accompanying=false delete_after_download=false delete_notifications=true dingtalk_token= directory_permissions=777 discord_id= discord_token= download_notifications=true download_path= file_permissions=666 folder_structure={:%Y/%m} gotify_app_token= gotify_https= gotify_server_url= group=group group_id=100 icloud_china= iyuu_token= jpeg_path= jpeg_quality=90 libraries_with_dates=true media_id_delete= media_id_download= media_id_expiration= media_id_startup= media_id_warning= nextcloud_delete=false nextcloud_password= nextcloud_upload=false nextcloud_url= nextcloud_username= notification_days=7 notification_type=Telegram photo_album= photo_library=SharedSync-########-####-####-####-############ photo_size=original prowl_api_key= pushover_sound=pushover pushover_token= pushover_user= recent_only= set_exif_datetime=false single_pass=false skip_album= skip_check=false skip_download=false skip_library= skip_live_photos=false skip_videos=false startup_notification=true synchronisation_delay=0 synchronisation_interval=86400 synology_ignore_path=false telegram_chat_id=##### telegram_polling=true telegram_server= telegram_silent_file_notifications= telegram_token=##### touser= trigger_nextlcoudcli_synchronisation= until_found= user_id=99 webhook_https=false webhook_id= webhook_path=/api/webhook/ webhook_port=8123 webhook_server= wecom_id= wecom_proxy= wecom_secret=

rustig136 commented 5 months ago

Thank you! Is your set up now storing the files in the folder structure? Is the folder structure {:%Y/%m} I got my Primary and Shared albums working but all files are downloaded in one of those two 'folders'. Like your post in issue #465

rustig136 commented 5 months ago

Pretty sure it's still this bit you need:

folder_structure={:%Y/%m/%d}

your config file has:

folder_structure= {:%Y/%m/%d}

@boredazfcuk You are correct; the system inadvertently inserted a space immediately after '='. When I created a new configuration from scratch, manually entering the parameters without copying, the system began functioning without generating that error. However, the files are not being stored in the anticipated date folder structure but all in one folder per album. Is that the expected behaviour?

Thank you.

blackfox33 commented 5 months ago

Thank you! Is your set up now storing the files in the folder structure? Is the folder structure {:%Y/%m} I got my Primary and Shared albums working but all files are downloaded in one of those two 'folders'. Like your post in issue #465

I set up 3 different containers. One for each personal library and one for the shared library. Everything is stored in year/month, so no issue there from my side. Only thing I want to get rid of is the additional subfolder with the name of the shared library.

rustig136 commented 5 months ago

hi @blackfox33

I'm confused and can't get it set up to way I would like it to use folder_structure={:%Y/%m} (for example). When I point the photo_library=SharedSync-########-####-####-####-############ it is always ignoring the folder_structure and put everything in the root of SharedSync-########-####-####-####-############.

I would love that the tool is creating the folder structure {:%Y/%m} or {:%Y/%m/%d} but it can't get it to work in combination with the photo_library parameter.

@boredazfcuk what I am experiencing, is this the expected behaviour or am I still doing something wrong? Even if I use photo_library="all libraries" it ignores the {:%Y/%m} folder structure.

Happy to hear from you both!

blackfox33 commented 5 months ago

hi @blackfox33

I'm confused and can't get it set up to way I would like it to use folder_structure={:%Y/%m} (for example). When I point the photo_library=SharedSync-########-####-####-####-############ it is always ignoring the folder_structure and put everything in the root of SharedSync-########-####-####-####-############.

I would love that the tool is creating the folder structure {:%Y/%m} or {:%Y/%m/%d} but it can't get it to work in combination with the photo_library parameter.

@boredazfcuk what I am experiencing, is this the expected behaviour or am I still doing something wrong? Even if I use photo_library="all libraries" it ignores the {:%Y/%m} folder structure.

Happy to hear from you both!

Strange behaviour, but I cannot tell where the problem lies. Other than the space between folder_structure and your setting, your config looks fine for me. In the configuration.md I found this:

folder_structure: This specifies the folder structure to use in your download destination directory. If this configuration option is not set, it will set {:%Y/%m/%d} as the default. Use none to download to a flat file structure.

Have you tried, switching to "none" for instance or leave it blank? In the last case it should match your desired folder structure, so no need to set {:%Y/%m/%d}.

rustig136 commented 5 months ago

@blackfox33, I omitted those parameters deliberately. For instance, when I didn't include "folder_structure," the system operated smoothly. It generated the expected results, but only if I refrained from specifying anything in "photo_library." Once I instruct the application to download from my "SharedSync" or "PrimarySync," it consolidates all files into a single folder.

I'm currently awaiting confirmation from @boredazfcuk on whether this behavior is expected or not.

Could you successfully download both of your libraries using a folder structure like "year-month," and if so, what steps did you take? I've only managed to make it work with my PrimarySync by excluding the "photo_library" parameter from the configuration.

rustig136 commented 5 months ago

Visit #465 for the solution.