search

boredazfcuk / docker-icloudpd

An Alpine Linux container for the iCloud Photos Downloader command line utility
1.59k stars 151 forks source link

After initialize, no files downloaded #408

Closed hollanbm closed 9 months ago

hollanbm commented 9 months ago

I just got the container setup, and initialized, then authed with MFA.

But I'm not getting any files downloaded. I'm not quite sure what I'm doing wrong, the timestamps also seem odd. The next run is scheduled in the past? The run time is usually only a few seconds

I've included a debug log, my config file, and docker-compose.yml

2023-10-08 02:57:36 INFO     ***** boredazfcuk/icloudpd container for icloud_photo_downloader v1.0.666 started *****
2023-10-08 02:57:36 INFO     ***** For support, please go here: https://github.com/boredazfcuk/docker-icloudpd *****
2023-10-08 02:57:36 INFO     Alpine Linux 3.18.3
2023-10-08 02:57:36 INFO     Python version: 3.11.5
2023-10-08 02:57:36 INFO     Loading configuration from: /config/icloudpd.conf
2023-10-08 02:57:36 DEBUG    Running user id: 0
2023-10-08 02:57:36 DEBUG    Running group id: 0
2023-10-08 02:57:36 DEBUG    Local user: hollanbm:1000
2023-10-08 02:57:36 DEBUG    Local group: hollanbm:1000
2023-10-08 02:57:36 DEBUG    Force GID: false
2023-10-08 02:57:36 DEBUG    LAN IP Address: 172.20.0.2
2023-10-08 02:57:36 DEBUG    Default gateway: 172.20.0.1
2023-10-08 02:57:36 DEBUG    DNS server: 127.0.0.11
2023-10-08 02:57:36 DEBUG    IP address for icloud.com: 17.253.144.10
2023-10-08 02:57:44 DEBUG    Route check to icloud.com successful
2023-10-08 02:57:44 DEBUG    Apple ID: (hidden)
2023-10-08 02:57:44 INFO     Authentication Type: MFA
2023-10-08 02:57:44 DEBUG    Cookie path: /config/(hidden)
2023-10-08 02:57:44 INFO     Cookie expiry notification period: 7
2023-10-08 02:57:44 INFO     Download destination directory: /home/hollanbm/iCloud/
2023-10-08 02:57:44 INFO     Folder structure: {:%Y/%m/%d}
2023-10-08 02:57:44 DEBUG    Directory permissions: 775
2023-10-08 02:57:44 DEBUG    File permissions: 0775
2023-10-08 02:57:44 INFO     Synchronisation interval: 86400
2023-10-08 02:57:44 INFO     Synchronisation delay (minutes): 0
2023-10-08 02:57:44 INFO     Set EXIF date/time: false
2023-10-08 02:57:44 INFO     Auto delete: true
2023-10-08 02:57:44 INFO     Delete after download: false
2023-10-08 02:57:44 INFO     Photo size: original
2023-10-08 02:57:44 INFO     Single pass mode: false
2023-10-08 02:57:44 INFO     Skip download check: true
2023-10-08 02:57:44 INFO     Skip live photos: false
2023-10-08 02:57:44 INFO     Number of most recently added photos to download: Download All Photos
2023-10-08 02:57:44 INFO     Downloading photos from album(s): all albums
2023-10-08 02:57:44 INFO     Stop downloading when prexisiting files count is: Download All Photos
2023-10-08 02:57:44 INFO     Live photo size: original
2023-10-08 02:57:44 INFO     Skip videos: false
2023-10-08 02:57:44 INFO     Convert HEIC to JPEG: false
2023-10-08 02:57:44 INFO     Pushover notifications enabled
2023-10-08 02:57:44 DEBUG    Notification title: boredazfcuk/iCloudPD
2023-10-08 02:57:44 DEBUG    Pushover user: (hidden)
2023-10-08 02:57:44 DEBUG    Pushover token: (hidden)
2023-10-08 02:57:44 DEBUG    Pushover sound: pushover
2023-10-08 02:57:44 INFO     Sending Pushover startup notification
2023-10-08 02:57:45 DEBUG    Pushover startup notification sent successfully
2023-10-08 02:57:45 DEBUG    Download notifications: Enabled
2023-10-08 02:57:45 DEBUG    Delete notifications: Enabled
2023-10-08 02:57:45 INFO     Downloading from: icloud.com
2023-10-08 02:57:45 INFO     Authentication domain: com
2023-10-08 02:57:45 DEBUG    Nextcloud synchronisation trigger: Disabled
2023-10-08 02:57:45 INFO     Nextcloud upload: Enabled
2023-10-08 02:57:45 INFO     Nextcloud URL: https://nextcloud.mydomain.com
2023-10-08 02:57:45 INFO     Nextcloud username: hollanbm
2023-10-08 02:57:45 INFO     Ignore Synology extended attribute directories: Disabled
2023-10-08 02:57:45 DEBUG    Group, hollanbm:1000, already created
2023-10-08 02:57:45 DEBUG    User, hollanbm:1000, already created
2023-10-08 02:57:45 DEBUG    Set owner and group on icloudpd temp directory
2023-10-08 02:57:45 DEBUG    Set owner and group on config directory
2023-10-08 02:57:45 INFO     Directory is writable: /config/python_keyring/
2023-10-08 02:57:45 DEBUG    Configure password
2023-10-08 02:57:45 DEBUG    Using password stored in keyring file: /config/python_keyring/keyring_pass.cfg
2023-10-08 02:57:45 INFO     Check download directory mounted correctly...
2023-10-08 02:57:45 INFO     Failsafe file /home/hollanbm/iCloud//.mounted exists, continuing
2023-10-08 02:57:45 DEBUG    Set owner and group on icloudpd temp directory
2023-10-08 02:57:45 DEBUG    Set owner and group on config directory
2023-10-08 02:57:45 INFO     Directory is writable: /config/python_keyring/
2023-10-08 02:57:45 INFO     Keyring file exists, continuing
2023-10-08 02:57:45 INFO     Sync user: hollanbm
2023-10-08 02:57:45 INFO     Synchronisation starting at 02:57:45
2023-10-08 02:57:45 INFO     Keyring file exists, continuing
2023-10-08 02:57:45 DEBUG    Check MFA Cookie
2023-10-08 02:57:45 DEBUG    Multifactor authentication cookie exists.
2023-10-08 02:57:45 DEBUG    Valid multifactor authentication cookie found. Days until expiration: 89
2023-10-08 02:57:45 INFO     Check download directory mounted correctly...
2023-10-08 02:57:45 INFO     Failsafe file /home/hollanbm/iCloud//.mounted exists, continuing
2023-10-08 02:57:45 DEBUG    Starting download of new files for user: hollanbm
2023-10-08 02:57:45 DEBUG    Downloading new files using password stored in keyring file...
2023-10-08 02:57:45 DEBUG    Switched to icloudpd: 1.16.2
ash: out of range
2023-10-08 02:57:50 DEBUG    Set owner on iCloud directory, if required
2023-10-08 02:57:50 DEBUG    Set group on iCloud directory, if required
2023-10-08 02:57:50 DEBUG    Set 775 permissions on iCloud directories, if required
2023-10-08 02:57:50 DEBUG    Set 0775 permissions on iCloud files, if required
2023-10-08 02:57:50 INFO     Synchronisation complete for hollanbm
2023-10-08 02:57:50 DEBUG    Web cookie exists.
2023-10-08 02:57:50 INFO     Web cookie expires: 2023-12-06 @ 20:52:10
2023-10-08 02:57:50 INFO     Multifactor authentication cookie expires: 2024-01-05 @ 20:52:33
2023-10-08 02:57:50 INFO     Days remaining until expiration: 89
2023-10-08 02:57:50 DEBUG    iCloud login counter = 1
2023-10-08 02:57:50 INFO     Synchronisation ended at 02:57:50
2023-10-08 02:57:50 INFO     Total time taken: 00:00:05
2023-10-08 02:57:50 INFO     Next synchronisation at 02:57:45

icloudpd.conf

albums_with_dates=false
apple_id=(hidden)
auth_china=false
authentication_type=MFA
auto_delete=true
bark_device_key=
bark_server=
convert_heic_to_jpeg=false
debug_logging=true
delete_accompanying=false
delete_after_download=false
delete_notifications=true
dingtalk_token=
directory_permissions=775
discord_id=
discord_token=
download_notifications=true
download_path=/home/hollanbm/iCloud
file_permissions=0775
folder_structure={:%Y/%m/%d}
gotify_app_token=
group_id=1000
group=hollanbm
icloud_china=
iyuu_token=
jpeg_path=
jpeg_quality=90
libraries_with_dates=false
live_photo_size=original
nextcloud_delete=false
nextcloud_password=(hidden)
nextcloud_target_dir=iCloud
nextcloud_upload=true
nextcloud_url=https://nextcloud.mydomain.com
nextcloud_username=hollanbm
notification_days=7
notification_type=Pushover
photo_album="all albums"
photo_library="all libraries"
photo_size=original
prowl_api_key=
pushover_sound=pushover
pushover_token=(hidden)
pushover_user=(hidden)
recent_only=
set_exif_datetime=false
single_pass=false
skip_check=true
skip_download=false
skip_library=
skip_live_photos=false
skip_videos=false
synchronisation_delay=0
synchronisation_interval=86400
synology_ignore_path=false
telegram_chat_id=
telegram_polling=false
telegram_server=
telegram_silent_file_notifications=
telegram_token=
trigger_nextlcoudcli_synchronisation=
TZ=America/Kentucky/Louisville
until_found=
user_id=1000
user=hollanbm
webhook_https=false
webhook_id=
webhook_path=/api/webhook/
webhook_port=8123
webhook_server=
wecom_id=
wecom_proxy=
wecom_secret=

docker-compose.yml

version: '3.8'
services:
  icloudpd:
    container_name: icloudpd
    image: boredazfcuk/icloudpd
    healthcheck:
      test: /usr/local/bin/healthcheck.sh
      start_period: 30s
    restart: always
    environment:
      - TZ=America/Kentucky/Louisville
    volumes:
      - icloud_nfs:/home/hollanbm/iCloud/
      - ${HOME:?err}/iCloudPD:/config

volumes:
  icloud_nfs:
    driver: local
    driver_opts:
      type: nfs
      o: addr=10.254.1.99,rw,noatime
      device: ":/mnt/RZ3/iCloud/"
boredazfcuk commented 9 months ago

This was a bug with the option to download from Photo Albums. Should be fixed in latest version. A word of warning though, if you downloading from more than one or two Photo Albums, you're gonna run into issues. Each album requires a new authentication attempt, when Apple receives too many in quick succession, it will force an MFA authentication, as it sees it as suspicious behaviour.

Also, I've noticed that you have this set:

photo_album="all albums"
photo_library="all libraries"

It can't be set to download from both Albums and Libraries at the same time. I didn't enable it, because I don't use either, and I'm not really sure how much they overlap.

This stems from the All Photos/Photo Album downloads. If somebody attempts to download "All Photos" and also "All Albums" they're going to end up with 3 copies of a lot of files.

hollanbm commented 9 months ago

This was a bug with the option to download from Photo Albums. Should be fixed in latest version. A word of warning though, if you downloading from more than one or two Photo Albums, you're gonna run into issues. Each album requires a new authentication attempt, when Apple receives too many in quick succession, it will force an MFA authentication, as it sees it as suspicious behaviour.

Also, I've noticed that you have this set:

photo_album="all albums"
photo_library="all libraries"

It can't be set to download from both Albums and Libraries at the same time. I didn't enable it, because I don't use either, and I'm not really sure how much they overlap.

This stems from the All Photos/Photo Album downloads. If somebody attempts to download "All Photos" and also "All Albums" they're going to end up with 3 copies of a lot of files.

I didn't realize that the library and albums would cause issues. That was indeed my issue.

Removing those two lines, and now it's syncing everything.