search

boredazfcuk / docker-icloudpd

An Alpine Linux container for the iCloud Photos Downloader command line utility
1.59k stars 151 forks source link

Failed to download new files and request entering iCloud password after restart icloudpd everytime #413

Closed tonyjohn2134 closed 9 months ago

tonyjohn2134 commented 9 months ago

After updating to the latest version, I have these three issues. Many thanks.

This is the first issue: I can backup the Shared Library, but not my own library. Is there any parameter I'm missing to config? And the folder_structure does not like {:%Y/%m/%d}, it just download all photos into a single folder /mnt/SharedSync-xxx/xxx.HEIC Can I config to /mnt/SharedSync-xxx/2023/10/13/xxx.HEIC and /mnt/PrimarySync/2023/10/13/xxx.HEIC?

This is the 2nd issue:

2023/10/13 04:52:06 | stdout | 2023-10-13 04:52:06 ERROR    ***** Please report problems here: https://github.com/boredazfcuk/docker-icloudpd/issues ***** -- | -- | -- 2023/10/13 04:52:06 | stdout | requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) 2023/10/13 04:52:06 | stdout | raise ConnectionError(err, request=request) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/requests/adapters.py", line 501, in send 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | r = adapter.send(request, **kwargs) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/requests/sessions.py", line 703, in send 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | resp = self.send(prep, **send_kwargs) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/requests/sessions.py", line 589, in request 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | response = super(PyiCloudSession, self).request(*args, **kwargs) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/pyicloud_ipd/base.py", line 71, in request 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | return self.request("POST", url, data=data, json=json, **kwargs) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/requests/sessions.py", line 637, in post 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | return self.service.session.post( 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/pyicloud_ipd/services/photos.py", line 337, in photos_request 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | request = self.photos_request(offset) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/pyicloud_ipd/services/photos.py", line 357, in photos 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | photos = (p for p in photos) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/icloudpd/base.py", line 834, in  2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | item = next(photos_iterator) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/icloudpd/base.py", line 889, in core 2023/10/13 04:52:06 | stdout | core( 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/icloudpd/base.py", line 317, in main 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | return __callback(*args, **kwargs) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/click/core.py", line 783, in invoke 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | return ctx.invoke(self.callback, **ctx.params) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/click/core.py", line 1434, in invoke 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | rv = self.invoke(ctx) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/click/core.py", line 1078, in main 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | return self.main(*args, **kwargs) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ 2023/10/13 04:52:06 | stdout | ^^^^^^ 2023/10/13 04:52:06 | stdout | sys.exit(main()) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/bin/icloudpd", line 8, in  2023/10/13 04:52:06 | stdout | Traceback (most recent call last): 2023/10/13 04:52:06 | stdout |   2023/10/13 04:52:06 | stdout | During handling of the above exception, another exception occurred: 2023/10/13 04:52:06 | stdout |   2023/10/13 04:52:06 | stdout | urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) 2023/10/13 04:52:06 | stdout | raise RemoteDisconnected("Remote end closed connection without" 2023/10/13 04:52:06 | stdout | File "/usr/lib/python3.11/http/client.py", line 287, in _read_status 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | version, status, reason = self._read_status() 2023/10/13 04:52:06 | stdout | File "/usr/lib/python3.11/http/client.py", line 318, in begin 2023/10/13 04:52:06 | stdout | response.begin() 2023/10/13 04:52:06 | stdout | File "/usr/lib/python3.11/http/client.py", line 1378, in getresponse 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | httplib_response = conn.getresponse() 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/connectionpool.py", line 461, in _make_request 2023/10/13 04:52:06 | stdout | File "", line 3, in raise_from 2023/10/13 04:52:06 | stdout | six.raise_from(e, None) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | httplib_response = self._make_request( 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/connectionpool.py", line 714, in urlopen 2023/10/13 04:52:06 | stdout | raise value.with_traceback(tb) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/packages/six.py", line 769, in reraise 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | raise six.reraise(type(error), error, _stacktrace) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/util/retry.py", line 550, in increment 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | retries = retries.increment( 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/connectionpool.py", line 798, in urlopen 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | resp = conn.urlopen( 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/requests/adapters.py", line 486, in send 2023/10/13 04:52:06 | stdout | Traceback (most recent call last): 2023/10/13 04:52:06 | stdout |   2023/10/13 04:52:06 | stdout | During handling of the above exception, another exception occurred: 2023/10/13 04:52:06 | stdout |   2023/10/13 04:52:06 | stdout | http.client.RemoteDisconnected: Remote end closed connection without response 2023/10/13 04:52:06 | stdout | raise RemoteDisconnected("Remote end closed connection without" 2023/10/13 04:52:06 | stdout | File "/usr/lib/python3.11/http/client.py", line 287, in _read_status 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | version, status, reason = self._read_status() 2023/10/13 04:52:06 | stdout | File "/usr/lib/python3.11/http/client.py", line 318, in begin 2023/10/13 04:52:06 | stdout | response.begin() 2023/10/13 04:52:06 | stdout | File "/usr/lib/python3.11/http/client.py", line 1378, in getresponse 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | httplib_response = conn.getresponse() 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/connectionpool.py", line 461, in _make_request 2023/10/13 04:52:06 | stdout | File "", line 3, in raise_from 2023/10/13 04:52:06 | stdout | six.raise_from(e, None) 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request 2023/10/13 04:52:06 | stdout | ^^^^^^^^^^^^^^^^^^^ 2023/10/13 04:52:06 | stdout | httplib_response = self._make_request( 2023/10/13 04:52:06 | stdout | File "/opt/icloudpd_latest/lib/python3.11/site-packages/urllib3/connectionpool.py", line 714, in urlopen 2023/10/13 04:52:06 | stdout | 2023-10-13 04:52:06 ERROR    Traceback (most recent call last): 2023/10/13 04:52:06 | stdout | 2023-10-13 04:52:06 ERROR    Error debugging info: 2023/10/13 04:52:06 | stdout | 2023-10-13 04:52:06 ERROR     - Can you log into icloud.com without receiving pop-up notifications? 2023/10/13 04:52:06 | stdout | 2023-10-13 04:52:06 ERROR    Failed to download new files

And this is third issue:

2023/10/13 10:18:42 | stdout | 2023-10-13 10:18:42 DEBUG    Looking up all photos and videos from album All Photos... -- | -- | -- 2023/10/13 10:18:36 | stdout | iCloud Password: 2023/10/13 10:18:18 | stdout | 2023-10-13 10:18:18 DEBUG    Authenticating... 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    iCloudPD launch command: /opt/icloudpd_latest/bin/icloudpd --directory /mnt --cookie-directory /config --domain com --username xxx@xxx --no-progress-bar --until-found 100 --folder-structure SharedSync-xxx --library SharedSync-xxx 2>/tmp/icloudpd/icloudpd_download_error 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Downloading library: SharedSync-xxx 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Starting Photo Library download 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Switched to icloudpd: 1.16.2 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Downloading new files using password stored in keyring file... 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Starting download of new files for user: xxx 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Failsafe file /mnt/.mounted exists, continuing 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Check download directory mounted correctly... 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Valid multifactor authentication cookie found. Days until expiration: 89 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Multifactor authentication cookie exists. 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Check MFA Cookie 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Keyring file exists, continuing 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Synchronisation starting at 10:18:17 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Sync user: xxx 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Keyring file exists, continuing 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Directory is writable: /config/python_keyring/ 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Set owner and group on config directory 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Set owner and group on icloudpd temp directory 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Failsafe file /mnt/.mounted exists, continuing 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Check download directory mounted correctly... 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Using password stored in keyring file: /config/python_keyring/keyring_pass.cfg 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Configure password 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Directory is writable: /config/python_keyring/ 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Set owner and group on config directory 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Set owner and group on icloudpd temp directory 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    User, xxx:xxx, already created 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Group, users:xxx, already created 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Ignore Synology extended attribute directories: Disabled 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Nextcloud upload: Disabled 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Nextcloud synchronisation trigger: Disabled 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Authentication domain: com 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Downloading from: icloud.com 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Convert HEIC to JPEG: false 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Skip videos: false 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Live photo size: original 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Stop downloading when prexisiting files count is: 100 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Downloading photos from library: SharedSync-xxx 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Number of most recently added photos to download: Download All Photos 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Skip live photos: false 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Skip download check: true 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Single pass mode: false 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Photo size: original 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Delete after download: false 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Auto delete: false 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Set EXIF date/time: false 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Synchronisation delay (minutes): 0 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Synchronisation interval: 86400 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    File permissions: 777 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Directory permissions: 777 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Folder structure: {:%Y/%m/%d} 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Download destination directory: /mnt 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Cookie expiry notification period: 7 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Cookie path: /config/(hidden) 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 INFO     Authentication Type: MFA 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Apple ID: (hidden) 2023/10/13 10:18:17 | stdout | 2023-10-13 10:18:17 DEBUG    Route check to icloud.com successful 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    IP address for icloud.com: xxx 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    DNS server: 192.168.1.1 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    Default gateway: xxx 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    LAN IP Address: xxx 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    Force GID: false 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    Local group: users:100 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    Local user: xxx:xxx 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    Running group id: 0 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 DEBUG    Running user id: 0 2023/10/13 10:18:09 | stdout | 2023-10-13 10:18:09 INFO     Loading configuration from: /config/icloudpd.conf 2023/10/13 10:18:08 | stdout | 2023-10-13 10:18:08 INFO     Python version: 3.11.5 2023/10/13 10:18:08 | stdout | 2023-10-13 10:18:08 INFO     Alpine Linux 3.18.3 2023/10/13 10:18:08 | stdout | 2023-10-13 10:18:08 INFO     ***** For support, please go here: https://github.com/boredazfcuk/docker-icloudpd ***** 2023/10/13 10:18:08 | stdout | 2023-10-13 10:18:08 INFO     ***** boredazfcuk/icloudpd container for icloud_photo_downloader v1.0.679 started *****

Many thanks for helping me.

ivanblack19 commented 9 months ago

Same issue

tonyjohn2134 commented 9 months ago

With ref with: https://github.com/boredazfcuk/docker-icloudpd/issues/393 After adding "XDG_DATA_HOME" and "/config" to environment variables, I can start the icloudpd without requesting iCloud password again.

But I still want to ask can I config to /mnt/SharedSync-xxx/2023/10/13/xxx.HEIC and /mnt/PrimarySync/2023/10/13/xxx.HEIC?

tonyjohn2134 commented 9 months ago

Sorry, I was missing to set "libraries_with_dates". All issues are solved.