search

boredazfcuk / docker-icloudpd

An Alpine Linux container for the iCloud Photos Downloader command line utility
1.85k stars 167 forks source link

KeyError: 'dsInfo'，but indicater“INFO Two factor authentication cookie generated. Sync should now be successful” #96

Closed Bamboowuzi closed 3 years ago

Bamboowuzi commented 3 years ago

i run it under unbuntu16.04:

root@R710-virtual-machine:/home# docker exec -it sync_icloud sync-icloud.sh --Initialise

2021-08-27 16:46:43 INFO boredazfcuk/icloudpd container for icloud_photo_downloader started 2021-08-27 16:46:43 INFO /usr/local/bin/sync-icloud.sh date: 2021/08/27_07:21 2021-08-27 16:46:43 INFO /usr/local/bin/sync-icloud.sh hash: 6455cb13ab5d02ee685aba70acd3f625 2021-08-27 16:46:43 INFO Alpine Linux 3.13.5 2021-08-27 16:46:43 INFO Python version: 3.8.10 2021-08-27 16:46:43 INFO icloudpd version: 1.7.2 2021-08-27 16:46:44 INFO pyicloud-ipd version: 0.10.1 2021-08-27 16:46:45 INFO Local user: r710:1000 2021-08-27 16:46:45 INFO Local group: r710:1000 2021-08-27 16:46:45 INFO Force GID: False 2021-08-27 16:46:45 INFO LAN IP Address: 172.17.0.4 2021-08-27 16:46:45 INFO Apple ID: xxx@gmail.com 2021-08-27 16:46:45 INFO Authentication Type: 2FA 2021-08-27 16:46:45 INFO Cookie path: /config/xxx 2021-08-27 16:46:45 INFO Cookie expiry notification period: 28 2021-08-27 16:46:45 INFO Download destination directory: /home/icloud 2021-08-27 16:46:45 INFO Folder structure: {:%Y} 2021-08-27 16:46:45 INFO Directory permissions: 750 2021-08-27 16:46:45 INFO File permissions: 640 2021-08-27 16:46:45 INFO Synchronisation interval: 86400 2021-08-27 16:46:45 INFO Synchronisation delay (minutes): 0 2021-08-27 16:46:45 INFO Time zone: Asia/Shanghai 2021-08-27 16:46:45 INFO Set EXIF date/time: False 2021-08-27 16:46:45 INFO Auto delete: True 2021-08-27 16:46:45 INFO Photo size: original 2021-08-27 16:46:45 INFO Skip download check: False 2021-08-27 16:46:45 INFO Skip live photos: False 2021-08-27 16:46:45 INFO Number of most recently added photos to download: Download All Photos 2021-08-27 16:46:45 INFO Stop downloading when prexisiting files count is: Download All Photos 2021-08-27 16:46:45 INFO Live photo size: original 2021-08-27 16:46:45 INFO Skip videos: False 2021-08-27 16:46:45 INFO Convert HEIC to JPEG: True 2021-08-27 16:46:45 INFO JPEG conversion quality: 100 2021-08-27 16:46:45 INFO Script launch parameters: --Initialise 2021-08-27 16:46:45 INFO Group, r710:1000, already created 2021-08-27 16:46:45 INFO User, r710:1000, already created 2021-08-27 16:46:45 INFO Correct owner on config directory, if required 2021-08-27 16:46:45 INFO Correct group on config directory, if required 2021-08-27 16:46:45 INFO Adding password to keyring file: /config/python_keyring/keyring_pass.cfg Enter iCloud password for xxx@gmail.com: Traceback (most recent call last): File "/usr/bin/icloud", line 33, in sys.exit(load_entry_point('pyicloud-ipd==0.10.1', 'console_scripts', 'icloud')()) File "/usr/lib/python3.8/site-packages/pyicloud_ipd/cmdline.py", line 200, in main api = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init self.authenticate() File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 219, in authenticate self.params.update({'dsid': resp['dsInfo']['dsid']}) KeyError: 'dsInfo' 2021-08-27 16:46:56 INFO Correct owner on config directory, if required 2021-08-27 16:46:56 INFO Correct group on config directory, if required 2021-08-27 16:46:56 INFO Generate 2FA cookie with password: 2021-08-27 16:46:56 INFO Check for new files using password stored in keyring file. iCloud Password: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/authentication.py", line 29, in authenticate icloud = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 141, in init password = get_password_from_keyring(apple_id) File "/usr/lib/python3.8/site-packages/pyicloud_ipd/utils.py", line 40, in get_password_from_keyring raise NoStoredPasswordAvailable( pyicloud_ipd.exceptions.NoStoredPasswordAvailable: No pyicloud password for xxx@gmail.com could be found in the system keychain. Use the --store-in-keyring command-line option for storing a password for this username.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/bin/icloudpd", line 33, in sys.exit(load_entry_point('icloudpd==1.7.2', 'console_scripts', 'icloudpd')()) File "/usr/lib/python3.8/site-packages/click/core.py", line 722, in call return self.main(args, kwargs) File "/usr/lib/python3.8/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/usr/lib/python3.8/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/lib/python3.8/site-packages/click/core.py", line 535, in invoke return callback(args, **kwargs) File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/base.py", line 255, in main icloud = authenticate( File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/authentication.py", line 36, in authenticate icloud = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init self.authenticate() File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 219, in authenticate self.params.update({'dsid': resp['dsInfo']['dsid']}) KeyError: 'dsInfo' 2021-08-27 16:47:06 INFO Two factor authentication cookie generated. Sync should now be successful. root@R710-virtual-machine:/home#

boredazfcuk commented 3 years ago

This is similar to an issue someone else reported on a Synology device. Can I ask what you're using?

The issue is that the file /config/python_keyring/keyring_pass.cfg has been created, but it is empty. It's possible that this happens if a password is not entered successfully first time around.

Can you ls -l your /config/python_keyring/ directory and let me know the size of the keyring_pass.cfg file please? Is it 0 bytes?

It could also be a permissions issue, that the file has been created, but the user account cannot read it, and results in the error.

Bamboowuzi commented 3 years ago

Thanks for your response. I use DELL R710 and run the container on ubuntu 16.04. i set my config file at /home/icloud/config, just i checked that there is no file keyring_pass.cfg been created . root@R710-virtual-machine:/home/icloud/config# root@R710-virtual-machine:/home/icloud/config# ll total 16 drwxrwxrwx 4 r710 r710 4096 8鏈 27 10:54 ./ drwxrwxrwx 5 root root 4096 8鏈 26 17:30 ../ drwxr-xr-x 2 root root 4096 8鏈 27 10:54 cookies/ drwxr-xr-x 2 r710 r710 4096 8鏈 26 17:32 python_keyring/ root@R710-virtual-machine:/home/icloud/config# cd python_keyring root@R710-virtual-machine:/home/icloud/config/python_keyring# ll total 8 drwxr-xr-x 2 r710 r710 4096 8鏈 26 17:32 ./ drwxrwxrwx 4 r710 r710 4096 8鏈 27 10:54 ../ root@R710-virtual-machine:/home/icloud/config/python_keyring#

boredazfcuk commented 3 years ago

Thats'a fair old machine for downloading iCloud photos. I've two here... dual 8-core CPUs, 256GB RAM... Had them in a cluster running about 40 VMs. Served me well.

Anyway... I've just pushed a new version which adds some logging information which will help me debug your problem. Can you remove your old container, create one from the new container version and then post the log again please?

Bamboowuzi commented 3 years ago

I creat a new container latest and log as below ,please check it ;thanks a lot root@R710-virtual-machine:/home# docker create \

--name sync_icloud \ --hostname sync_icloud \ --network bridge \ --restart=always \ --env user=r710 \ --env user_id=1000 \ --env group=r710 \ --env group_id=1000 \ --env apple_id=bamboowuzi@gmail.com \ --env authentication_type=2FA \ --env folder_structure={:%Y} \ --env download_path=/home/icloud/F \ --env auto_delete=True \ --env notification_days=28 \ --env synchronisation_interval=21600 \ --env TZ=Asia/Shanghai \ --env convert_heic_to_jpeg=True \ --env jpeg_quality=100 \ --volume /home/icloud/config:/config \ --volume /home/icloud:/home/boredazfcuk/iCloud \ boredazfcuk/icloudpd Unable to find image 'boredazfcuk/icloudpd:latest' locally latest: Pulling from boredazfcuk/icloudpd 540db60ca938: Pull complete 5bb2c0264cb6: Pull complete 0fb1b473be2e: Pull complete 7411aaacf2c2: Pull complete Digest: sha256:833132d5e5c6d80d912a7675e90aa6645fc8f7e9e4c402650bb4e233bbcdb9c7 Status: Downloaded newer image for boredazfcuk/icloudpd:latest ef528e775ab34caef6cd905591e05714c2111c842914758c2ee129f0797d3a9d root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ef528e775ab3 boredazfcuk/icloudpd "/bin/sh -c /usr/loc鈥 9 seconds ago Created sync_icloud 604d24ae25bf portainer/portainer "/portainer" 19 hours ago Up 19 hours 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp portainer c2abd0178c8f linkease/ddnsto "/usr/bin/ddnsto-mon鈥 2 weeks ago Up 23 hours ddnsto root@R710-virtual-machine:/home# docker exec -it sync_icloud sync-icloud.sh --Initialise Error response from daemon: Container ef528e775ab34caef6cd905591e05714c2111c842914758c2ee129f0797d3a9d is not running root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# docker exec -it sync_icloud sync-icloud.sh --Initialise

2021-09-03 09:34:56 INFO boredazfcuk/icloudpd container for icloud_photo_downloader started 2021-09-03 09:34:56 INFO /usr/local/bin/sync-icloud.sh date: 2021/09/02_22:29 2021-09-03 09:34:56 INFO /usr/local/bin/sync-icloud.sh hash: 6f366e6533d0c89b0076318a4e73f149 2021-09-03 09:34:56 INFO Alpine Linux 3.13.5 2021-09-03 09:34:56 INFO Python version: 3.8.10 2021-09-03 09:34:56 INFO icloudpd version: 1.7.2 2021-09-03 09:34:57 INFO pyicloud-ipd version: 0.10.1 2021-09-03 09:34:59 INFO Running user id: 0 2021-09-03 09:34:59 INFO Running group id: 0 2021-09-03 09:34:59 INFO Local user: r710:1000 2021-09-03 09:34:59 INFO Local group: r710:1000 2021-09-03 09:34:59 INFO Force GID: False 2021-09-03 09:34:59 INFO LAN IP Address: 172.17.0.3 2021-09-03 09:34:59 INFO Apple ID: bamboowuzi@gmail.com 2021-09-03 09:34:59 INFO Authentication Type: 2FA 2021-09-03 09:34:59 INFO Cookie path: /config/bamboowuzigmailcom 2021-09-03 09:34:59 INFO Cookie expiry notification period: 28 2021-09-03 09:34:59 INFO Download destination directory: /home/icloud/F 2021-09-03 09:34:59 INFO Folder structure: {:%Y} 2021-09-03 09:34:59 INFO Directory permissions: 750 2021-09-03 09:34:59 INFO File permissions: 640 2021-09-03 09:34:59 INFO Synchronisation interval: 86400 2021-09-03 09:34:59 INFO Synchronisation delay (minutes): 0 2021-09-03 09:34:59 INFO Time zone: Asia/Shanghai 2021-09-03 09:34:59 INFO Set EXIF date/time: False 2021-09-03 09:34:59 INFO Auto delete: True 2021-09-03 09:34:59 INFO Photo size: original 2021-09-03 09:34:59 INFO Skip download check: False 2021-09-03 09:34:59 INFO Skip live photos: False 2021-09-03 09:34:59 INFO Number of most recently added photos to download: Download All Photos 2021-09-03 09:34:59 INFO Stop downloading when prexisiting files count is: Download All Photos 2021-09-03 09:34:59 INFO Live photo size: original 2021-09-03 09:34:59 INFO Skip videos: False 2021-09-03 09:34:59 INFO Convert HEIC to JPEG: True 2021-09-03 09:34:59 INFO JPEG conversion quality: 100 2021-09-03 09:34:59 INFO Script launch parameters: --Initialise 2021-09-03 09:34:59 INFO Group, r710:1000, already created 2021-09-03 09:34:59 INFO User, r710:1000, already created 2021-09-03 09:34:59 INFO Correct owner on config directory, if required 2021-09-03 09:34:59 INFO Correct group on config directory, if required 2021-09-03 09:34:59 INFO Adding password to keyring file: /config/python_keyring/keyring_pass.cfg Enter iCloud password for bamboowuzi@gmail.com: Traceback (most recent call last): File "/usr/bin/icloud", line 33, in sys.exit(load_entry_point('pyicloud-ipd==0.10.1', 'console_scripts', 'icloud')()) File "/usr/lib/python3.8/site-packages/pyicloud_ipd/cmdline.py", line 200, in main api = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init self.authenticate() File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 219, in authenticate self.params.update({'dsid': resp['dsInfo']['dsid']}) KeyError: 'dsInfo' 2021-09-03 09:35:11 INFO Correct owner on config directory, if required 2021-09-03 09:35:11 INFO Correct group on config directory, if required 2021-09-03 09:35:11 INFO Generate 2FA cookie with password: 2021-09-03 09:35:11 INFO Check for new files using password stored in keyring file. iCloud Password: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/authentication.py", line 29, in authenticate icloud = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 141, in init password = get_password_from_keyring(apple_id) File "/usr/lib/python3.8/site-packages/pyicloud_ipd/utils.py", line 40, in get_password_from_keyring raise NoStoredPasswordAvailable( pyicloud_ipd.exceptions.NoStoredPasswordAvailable: No pyicloud password for bamboowuzi@gmail.com could be found in the system keychain. Use the --store-in-keyring command-line option for storing a password for this username.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/bin/icloudpd", line 33, in sys.exit(load_entry_point('icloudpd==1.7.2', 'console_scripts', 'icloudpd')()) File "/usr/lib/python3.8/site-packages/click/core.py", line 722, in call return self.main(args, kwargs) File "/usr/lib/python3.8/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/usr/lib/python3.8/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/lib/python3.8/site-packages/click/core.py", line 535, in invoke return callback(args, **kwargs) File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/base.py", line 255, in main icloud = authenticate( File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/authentication.py", line 36, in authenticate icloud = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init self.authenticate() File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 219, in authenticate self.params.update({'dsid': resp['dsInfo']['dsid']}) KeyError: 'dsInfo' 2021-09-03 09:35:23 INFO Two factor authentication cookie generated. Sync should now be successful. root@R710-virtual-machine:/home#

boredazfcuk commented 3 years ago

Hi, Sorry, life's been a bit busy recently...

It looks like ther are a couple of odd things here.

In your creation command, you map a volume:

--volume /home/icloud:/home/boredazfcuk/iCloud

The path inside your container /home/boredazfcuk/iCloud won't be accessed by the script as your user is set to r710. It will only ever access /home/710/iCloud

Also, you are configuring your download path as:

download_path=/home/icloud/F

So you should really mount your volume to /home/icloud/F if you're using this.

This bit also looks a bit messy too:

--volume /home/icloud/config:/config
--volume /home/icloud:/home/boredazfcuk/iCloud

It means the path on your host machine /home/icloud/config lives in /config... The problem is, that you're then mapping the parent directory /home/icloud to /home/boredazfcuk/iCloud.

I'm not sure how Docker would handle this... Does your /home/icloud/config folder appear twice inside the container (/config and /home/boredazfcuk/config)? Does it invalidate the first mapping?

I'd try creating your container differently... You can leave out all the options which are default values... So something like this:

docker create \
    --name sync_icloud
    --hostname sync_icloud
    --network bridge
    --restart=always
    --env user=r710
    --env group=r710
    --env apple_id=***@gmail.com
    --env folder_structure={:%Y}
    --env auto_delete=True
    --env notification_days=28
    --env synchronisation_interval=43200
    --env TZ=Asia/Shanghai
    --env convert_heic_to_jpeg=True
    --env jpeg_quality=100
    --volume /home/icloud/config:/config
    --volume /home/icloud/photos:/home/r710/iCloud
    boredazfcuk/icloudpd

Making sure you create the directory /home/icloud/photos on your host machine.

Give that a go and see what happens.

Please note, your synchronisation_interval value of 21600 is invalid. This will set it the default instead 86400 (24hrs). The lowest value you can set it to is 43200 (12hrs).

Bamboowuzi commented 3 years ago

sorry ,a little busy recently also. I copy your command and create the container again as below,but it seems same thing. i'm sure the volume mapping is no problem and i verfiy it in the container inside root@R710-virtual-machine:/home# docker create \

--name sync_icloud\
--hostname sync_icloud\
--network bridge\
--restart=always\
--env user=r710\
--env group=r710\
--env apple_id=bamboowuzi@gmail.com\
--env folder_structure={:%Y}\
--env auto_delete=True\
--env notification_days=28\
--env synchronisation_interval=43200\
--env TZ=Asia/Shanghai\
--env convert_heic_to_jpeg=True\
--env jpeg_quality=100\
--volume /home/icloud/config:/config\
--volume /home/icloud/F:/home/r710/iCloud\
boredazfcuk/icloudpd

ea3fd8e8f32454bdb1fd0f2cb69977e7df17e0e101a9f747dfdd8af470a66bf3 root@R710-virtual-machine:/home# docker exec -it sync_icloud sync-icloud.sh --Initialise

2021-09-15 16:14:02 INFO boredazfcuk/icloudpd container for icloud_photo_downloader started 2021-09-15 16:14:02 INFO /usr/local/bin/sync-icloud.sh date: 2021/09/13_17:25 2021-09-15 16:14:02 INFO /usr/local/bin/sync-icloud.sh hash: 4a21f2ba5f89d71f406a408242f55179 2021-09-15 16:14:02 INFO Alpine Linux 3.13.5 2021-09-15 16:14:02 INFO Python version: 3.8.10 2021-09-15 16:14:02 INFO icloudpd version: 1.7.2 2021-09-15 16:14:03 INFO pyicloud-ipd version: 0.10.1 2021-09-15 16:14:04 INFO Running user id: 0 2021-09-15 16:14:04 INFO Running group id: 0 2021-09-15 16:14:04 INFO Local user: r710:1000 2021-09-15 16:14:04 INFO Local group: r710:1000 2021-09-15 16:14:04 INFO Force GID: False 2021-09-15 16:14:04 INFO LAN IP Address: 172.17.0.2 2021-09-15 16:14:04 INFO Apple ID: bamboowuzi@gmail.com 2021-09-15 16:14:04 INFO Authentication Type: 2FA 2021-09-15 16:14:04 INFO Cookie path: /config/bamboowuzigmailcom 2021-09-15 16:14:04 INFO Cookie expiry notification period: 28 2021-09-15 16:14:04 INFO Download destination directory: /home/r710/iCloud 2021-09-15 16:14:04 INFO Folder structure: {:%Y} 2021-09-15 16:14:04 INFO Directory permissions: 750 2021-09-15 16:14:04 INFO File permissions: 640 2021-09-15 16:14:04 INFO Synchronisation interval: 43200 2021-09-15 16:14:04 INFO Synchronisation delay (minutes): 0 2021-09-15 16:14:04 INFO Time zone: Asia/Shanghai 2021-09-15 16:14:04 INFO Set EXIF date/time: False 2021-09-15 16:14:04 INFO Auto delete: True 2021-09-15 16:14:04 INFO Photo size: original 2021-09-15 16:14:04 INFO Skip download check: False 2021-09-15 16:14:04 INFO Skip live photos: False 2021-09-15 16:14:04 INFO Number of most recently added photos to download: Download All Photos 2021-09-15 16:14:04 INFO Stop downloading when prexisiting files count is: Download All Photos 2021-09-15 16:14:04 INFO Live photo size: original 2021-09-15 16:14:04 INFO Skip videos: False 2021-09-15 16:14:04 INFO Convert HEIC to JPEG: True 2021-09-15 16:14:04 INFO JPEG conversion quality: 100 2021-09-15 16:14:04 INFO Script launch parameters: --Initialise 2021-09-15 16:14:04 INFO Group, r710:1000, already created 2021-09-15 16:14:04 INFO User, r710:1000, already created 2021-09-15 16:14:04 INFO Set owner, r710, on iCloud directory, if required 2021-09-15 16:14:04 INFO Set group, r710, on iCloud directory, if required 2021-09-15 16:14:04 INFO Correct owner on icloudpd temp directory, if required 2021-09-15 16:14:04 INFO Correct group on icloudpd temp directory, if required 2021-09-15 16:14:04 INFO Correct owner on config directory, if required 2021-09-15 16:14:04 INFO Correct group on config directory, if required 2021-09-15 16:14:04 INFO Correct owner on keyring directory, if required 2021-09-15 16:14:04 INFO Correct group on keyring directory, if required 2021-09-15 16:14:04 INFO Set 750 permissions on iCloud directories, if required 2021-09-15 16:14:04 INFO Set 640 permissions on iCloud files, if required 2021-09-15 16:14:04 INFO Configure password 2021-09-15 16:14:04 INFO Adding password to keyring file: /config/python_keyring/keyring_pass.cfg Enter iCloud password for bamboowuzi@gmail.com: Traceback (most recent call last): File "/usr/bin/icloud", line 33, in sys.exit(load_entry_point('pyicloud-ipd==0.10.1', 'console_scripts', 'icloud')()) File "/usr/lib/python3.8/site-packages/pyicloud_ipd/cmdline.py", line 200, in main api = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init self.authenticate() File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 219, in authenticate self.params.update({'dsid': resp['dsInfo']['dsid']}) KeyError: 'dsInfo' 2021-09-15 16:14:15 INFO Correct owner on config directory, if required 2021-09-15 16:14:15 INFO Correct group on config directory, if required 2021-09-15 16:14:15 INFO Generate 2FA cookie using password stored in keyring file. iCloud Password: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/authentication.py", line 29, in authenticate icloud = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 141, in init password = get_password_from_keyring(apple_id) File "/usr/lib/python3.8/site-packages/pyicloud_ipd/utils.py", line 40, in get_password_from_keyring raise NoStoredPasswordAvailable( pyicloud_ipd.exceptions.NoStoredPasswordAvailable: No pyicloud password for bamboowuzi@gmail.com could be found in the system keychain. Use the --store-in-keyring command-line option for storing a password for this username.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/bin/icloudpd", line 33, in sys.exit(load_entry_point('icloudpd==1.7.2', 'console_scripts', 'icloudpd')()) File "/usr/lib/python3.8/site-packages/click/core.py", line 722, in call return self.main(args, kwargs) File "/usr/lib/python3.8/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/usr/lib/python3.8/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/lib/python3.8/site-packages/click/core.py", line 535, in invoke return callback(args, **kwargs) File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/base.py", line 255, in main icloud = authenticate( File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/authentication.py", line 36, in authenticate icloud = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init self.authenticate() File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 219, in authenticate self.params.update({'dsid': resp['dsInfo']['dsid']}) KeyError: 'dsInfo' 2021-09-15 16:14:52 INFO Two factor authentication cookie generated. Sync should now be successful. root@R710-virtual-machine:/home# ll total 28 drwxr-xr-x 7 root root 4096 9鏈 2 12:08 ./ drwxr-xr-x 25 root root 4096 9鏈 2 12:15 ../ drwxrwxrwx 2 root root 4096 9鏈 13 18:50 download/ drwxrwxrwx 5 root root 4096 8鏈 26 17:30 icloud/ drwxr-xr-x 8 root root 4096 8鏈 24 13:10 plex/ drwxrwxrwx 2 root root 4096 9鏈 7 16:01 public/ drwxr-xr-x 17 r710 r710 4096 9鏈 14 09:03 r710/ root@R710-virtual-machine:/home# cd icloud root@R710-virtual-machine:/home/icloud# ll total 48 drwxrwxrwx 5 root root 4096 8鏈 26 17:30 ./ drwxr-xr-x 7 root root 4096 9鏈 2 12:08 ../ drwxrwxrwx 4 r710 r710 4096 8鏈 27 10:54 config/ drwxr-x--- 2 r710 r710 20480 9鏈 3 17:34 F/ drwxrwxrwx 2 root root 16384 9鏈 2 17:00 R/ root@R710-virtual-machine:/home/icloud# ll config total 16 drwxrwxrwx 4 r710 r710 4096 8鏈 27 10:54 ./ drwxrwxrwx 5 root root 4096 8鏈 26 17:30 ../ drwxr-xr-x 2 r710 r710 4096 8鏈 27 10:54 cookies/ drwxrwxrwx 2 r710 r710 4096 9鏈 2 09:35 python_keyring/ root@R710-virtual-machine:/home/icloud#

boredazfcuk commented 3 years ago

Do you have any non-alphanumeric characters in your password that may mess up Linux? Like: ’”[]{}?* etc.?

boredazfcuk commented 3 years ago

i run it under unbuntu16.04:

Also, can you post the results of docker version when run on the host machine? It should look something like this:

Client: Docker Engine - Community
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:47 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:54:58 2021
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.4.8
  GitCommit:        7eba5930496d9bbe375fdf71603e610ad737d2b2
 runc:
  Version:          1.0.0
  GitCommit:        v1.0.0-0-g84113ee
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
Bamboowuzi commented 3 years ago

my password is alphanumeric and no specidal characters. and another thing is my icoud runs on guizhou of china

root@R710-virtual-machine:/home/r710# docker version Client: Docker Engine - Community Version: 20.10.7 API version: 1.41 Go version: go1.13.15 Git commit: f0df350 Built: Wed Jun 2 11:56:47 2021 OS/Arch: linux/amd64 Context: default Experimental: true

Server: Docker Engine - Community Engine: Version: 20.10.7 API version: 1.41 (minimum version 1.12) Go version: go1.13.15 Git commit: b0f5bc3 Built: Wed Jun 2 11:54:58 2021 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.4.6 GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d runc: Version: 1.0.0-rc95 GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7 docker-init: Version: 0.19.0 GitCommit: de40ad0 root@R710-virtual-machine:/home/r710#

boredazfcuk commented 3 years ago

Can you create a dedicated network for the container and then attach it to that network? It looks like you're using the default network, which according to the Docker documentation, doesn't allow DNS resolution, or external traffic. (https://docs.docker.com/network/bridge/)

So, something like:

docker network create \
   --driver=bridge \
   --subnet=192.168.115.0/24 \
   --gateway=192.168.115.254 \
   --opt com.docker.network.bridge.name=icloudpd_br0 \
   icloudpd_bridge

then

docker create \
    --name sync_icloud
    --hostname sync_icloud
    --network icloudpd_bridge
    --restart=always
    --env user=r710
    --env group=r710
    --env apple_id=***@gmail.com
    --env folder_structure={:%Y}
    --env auto_delete=True
    --env notification_days=28
    --env synchronisation_interval=43200
    --env TZ=Asia/Shanghai
    --env convert_heic_to_jpeg=True
    --env jpeg_quality=100
    --volume /home/icloud/config:/config
    --volume /home/icloud/photos:/home/r710/iCloud
    boredazfcuk/icloudpd
Bamboowuzi commented 3 years ago

i try it according to your suggestion,but seems same: root@R710-virtual-machine:/home# docker network create \

--driver=bridge \ --subnet=192.168.115.0/24 \ --gateway=192.168.115.254 \ --opt com.docker.network.bridge.name=icloudpd_br0 \ icloudpd_bridge a827072ebb55066e4a8f86a8cafc9b6f2e31ca5777e3c3a23d8001a61f454fbd root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# docker create \ --name sync_icloud \ --hostname sync_icloud \ --network icloudpd_bridge \ --restart=always \ --env user=r710 \ --env group=r710 \ --env apple_id=bamboowuzi@gmail.com \ --env folder_structure={:%Y} \ --env auto_delete=True \ --env notification_days=28 \ --env synchronisation_interval=43200 \ --env TZ=Asia/Shanghai \ --env convert_heic_to_jpeg=True \ --env jpeg_quality=100 \ --volume /home/icloud/config:/config \ --volume /home/icloud/F:/home/r710/iCloud \ boredazfcuk/icloudpd Unable to find image 'boredazfcuk/icloudpd:latest' locally latest: Pulling from boredazfcuk/icloudpd 540db60ca938: Pull complete 6c2af49a26ad: Pull complete 45667911aeaf: Pull complete 657af178745a: Pull complete Digest: sha256:45293fd8146d8d40080b75d416aaece9630f31ba2aa92e9b0ebf444feb07e0ee Status: Downloaded newer image for boredazfcuk/icloudpd:latest 907dc0799e48a091ea3140a0e416afee44a27e208991af0b8deb1fd9e9d61728 root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# root@R710-virtual-machine:/home# docker exec -it sync_icloud sync-icloud.sh --Initialise

2021-09-23 15:01:16 INFO boredazfcuk/icloudpd container for icloud_photo_downloader started 2021-09-23 15:01:16 INFO /usr/local/bin/sync-icloud.sh date: 2021/09/21_19:45 2021-09-23 15:01:16 INFO /usr/local/bin/sync-icloud.sh hash: 4f7994ad43454bba6aabe4fc46788e50 2021-09-23 15:01:16 INFO Alpine Linux 3.13.5 2021-09-23 15:01:16 INFO Python version: 3.8.10 2021-09-23 15:01:16 INFO icloudpd version: 1.7.2 2021-09-23 15:01:17 INFO pyicloud-ipd version: 0.10.1 2021-09-23 15:01:18 INFO Running user id: 0 2021-09-23 15:01:18 INFO Running group id: 0 2021-09-23 15:01:18 INFO Local user: r710:1000 2021-09-23 15:01:18 INFO Local group: r710:1000 2021-09-23 15:01:18 INFO Force GID: False 2021-09-23 15:01:18 INFO LAN IP Address: 192.168.115.1 2021-09-23 15:01:18 INFO Default gateway: 192.168.115.254 2021-09-23 15:01:18 INFO DNS server: 127.0.0.11 2021-09-23 15:01:18 INFO DNS lookup for icloud.com: 17.253.144.10 2021-09-23 15:01:23 INFO Route check to icloud.com successful 2021-09-23 15:01:23 INFO Apple ID: bamboowuzi@gmail.com 2021-09-23 15:01:23 INFO Authentication Type: 2FA 2021-09-23 15:01:23 INFO Cookie path: /config/bamboowuzigmailcom 2021-09-23 15:01:23 INFO Cookie expiry notification period: 28 2021-09-23 15:01:23 INFO Download destination directory: /home/r710/iCloud 2021-09-23 15:01:23 INFO Folder structure: {:%Y} 2021-09-23 15:01:23 INFO Directory permissions: 750 2021-09-23 15:01:23 INFO File permissions: 640 2021-09-23 15:01:23 INFO Synchronisation interval: 43200 2021-09-23 15:01:23 INFO Synchronisation delay (minutes): 0 2021-09-23 15:01:23 INFO Time zone: Asia/Shanghai 2021-09-23 15:01:23 INFO Set EXIF date/time: False 2021-09-23 15:01:23 INFO Auto delete: True 2021-09-23 15:01:23 INFO Photo size: original 2021-09-23 15:01:23 INFO Skip download check: False 2021-09-23 15:01:23 INFO Skip live photos: False 2021-09-23 15:01:23 INFO Number of most recently added photos to download: Download All Photos 2021-09-23 15:01:23 INFO Stop downloading when prexisiting files count is: Download All Photos 2021-09-23 15:01:23 INFO Live photo size: original 2021-09-23 15:01:23 INFO Skip videos: False 2021-09-23 15:01:23 INFO Convert HEIC to JPEG: True 2021-09-23 15:01:23 INFO JPEG conversion quality: 100 2021-09-23 15:01:23 INFO Script launch parameters: --Initialise 2021-09-23 15:01:23 INFO Group, r710:1000, already created 2021-09-23 15:01:23 INFO User, r710:1000, already created 2021-09-23 15:01:23 INFO Set owner, r710, on iCloud directory, if required 2021-09-23 15:01:23 INFO Set group, r710, on iCloud directory, if required 2021-09-23 15:01:23 INFO Correct owner on icloudpd temp directory, if required 2021-09-23 15:01:23 INFO Correct group on icloudpd temp directory, if required 2021-09-23 15:01:23 INFO Correct owner on config directory, if required 2021-09-23 15:01:23 INFO Correct group on config directory, if required 2021-09-23 15:01:23 INFO Correct owner on keyring directory, if required 2021-09-23 15:01:23 INFO Correct group on keyring directory, if required 2021-09-23 15:01:23 INFO Set 750 permissions on iCloud directories, if required 2021-09-23 15:01:23 INFO Set 640 permissions on iCloud files, if required 2021-09-23 15:01:23 INFO Configure password 2021-09-23 15:01:23 INFO Adding password to keyring file: /config/python_keyring/keyring_pass.cfg Enter iCloud password for bamboowuzi@gmail.com: Traceback (most recent call last): File "/usr/bin/icloud", line 33, in sys.exit(load_entry_point('pyicloud-ipd==0.10.1', 'console_scripts', 'icloud')()) File "/usr/lib/python3.8/site-packages/pyicloud_ipd/cmdline.py", line 200, in main api = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init self.authenticate() File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 219, in authenticate self.params.update({'dsid': resp['dsInfo']['dsid']}) KeyError: 'dsInfo' 2021-09-23 15:01:33 INFO Correct owner on config directory, if required 2021-09-23 15:01:33 INFO Correct group on config directory, if required 2021-09-23 15:01:33 INFO Generate 2FA cookie using password stored in keyring file. iCloud Password: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/authentication.py", line 29, in authenticate icloud = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 141, in init password = get_password_from_keyring(apple_id) File "/usr/lib/python3.8/site-packages/pyicloud_ipd/utils.py", line 40, in get_password_from_keyring raise NoStoredPasswordAvailable( pyicloud_ipd.exceptions.NoStoredPasswordAvailable: No pyicloud password for bamboowuzi@gmail.com could be found in the system keychain. Use the --store-in-keyring command-line option for storing a password for this username.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/bin/icloudpd", line 33, in sys.exit(load_entry_point('icloudpd==1.7.2', 'console_scripts', 'icloudpd')()) File "/usr/lib/python3.8/site-packages/click/core.py", line 722, in call return self.main(args, kwargs) File "/usr/lib/python3.8/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/usr/lib/python3.8/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/lib/python3.8/site-packages/click/core.py", line 535, in invoke return callback(args, **kwargs) File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/base.py", line 255, in main icloud = authenticate( File "/usr/lib/python3.8/site-packages/icloudpd-1.7.2-py3.8.egg/icloudpd/authentication.py", line 36, in authenticate icloud = pyicloud_ipd.PyiCloudService( File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init self.authenticate() File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 219, in authenticate self.params.update({'dsid': resp['dsInfo']['dsid']}) KeyError: 'dsInfo' 2021-09-23 15:03:05 INFO Two factor authentication cookie generated. Sync should now be successful. root@R710-virtual-machine:/home#

boredazfcuk commented 3 years ago

I can see in your log that DNS is resolving OK and that it can connect to iCloud:

2021-09-23 15:01:18 INFO DNS lookup for icloud.com: 17.253.144.10
2021-09-23 15:01:23 INFO Route check to icloud.com successful

So the problem must be related to the communication between the container and iCloud.

Youre account's not locked out is it? What happens when you go to icloud.com and login in manually? Does it ask you to re-authorise your account by answering the security questions? Also, you do get prompted to confirm your connection with the 2FA process? Just wondering if you have 2FA enabled on a non-2FA account?

What's in your cookie file, /config/bamboowuzigmailcom? There should be one or two lines which start with:

Set-Cookie3: X-APPLE-WEBAUTH-HSA-TRUST-<Sensitive Info Removed>
Set-Cookie3: X_APPLE_WEB_KB-<Sensitive Info Removed>

The Set-Cookie3: X_APPLE_WEB_KB- line should appear as soon as the container conencts to iCloud and the Set-Cookie3: X-APPLE-WEBAUTH-HSA-TRUST- should appear as soon as 2FA authentication is successful (I'm guessing you don't have this one).

Also can you connect to the container with:

docker exec -it --user r710 sync_icloud /bin/ash

This will connect you as the user which runs the connection utility. Make sure you can move to all the following directories and list the files in there (and see you're the owner, check permissions etc):

cd  /config
ls -al
cd /home/r710/iCloud
ls -al
cd /config/python_keyring
ls -al
cd /home/${user}/.local/share/python_keyring
ls -al
boredazfcuk commented 3 years ago

Closing this off as I can't replicate it and this thread seems to have been abandoned.

boredazfcuk commented 2 years ago

Just in case you are still monitoring this thread, I believe that the source of your problem has been identified in this upstream thread: https://github.com/icloud-photos-downloader/icloud_photos_downloader/issues/392

Somebody else logged the issue against my container, so I've implemented a fix. See the issue here for further details: https://github.com/boredazfcuk/docker-icloudpd/issues/117