search

mandarons / icloud-docker

Dockerized iCloud Client - make a local copy of your iCloud documents and photos, and keep it automatically up-to-date.
BSD 3-Clause "New" or "Revised" License
1.19k stars 49 forks source link

[BUG] Bad Request (400) when using no filters in config for drive #113

Closed nicx closed 1 year ago

nicx commented 1 year ago

Describe the bug I just switched my config from using a filter for drive sync to not using any filters. I just wanted to sync all drive files and folders. But with the new config I get an error and the container stops.

logfile:

Loading config from /app/config.yaml ...
Traceback (most recent call last):
  File "/app/./src/main.py", line 7, in <module>
2023-01-12 08:50:00,518 :: ERROR :: icloudpy.base :: base.py :: 186 :: Bad Request (400)
    sync.sync()
  File "/app/src/sync.py", line 72, in sync
    sync_drive.sync_drive(config=config, drive=api.drive)
  File "/app/src/sync_drive.py", line 292, in sync_drive
    return sync_directory(
  File "/app/src/sync_drive.py", line 259, in sync_directory
    sync_directory(
  File "/app/src/sync_drive.py", line 262, in sync_directory
    items=item.dir(),
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 323, in dir
    return [child.name for child in self.get_children()]
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 265, in get_children
    self.data.update(self.connection.get_node_data(self.data["drivewsid"]))
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 34, in get_node_data
    request = self.session.post(
  File "/app/venv/lib/python3.10/site-packages/requests/sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/icloudpy/base.py", line 158, in request
    self._raise_error(code, reason)
  File "/app/venv/lib/python3.10/site-packages/icloudpy/base.py", line 187, in _raise_error
    raise api_error
icloudpy.exceptions.ICloudPyAPIResponseException: Bad Request (400)

To Reproduce Hash filter options in config.yaml, then restart docker container.

Expected behavior No crash :)

Configuration old config:

drive:
  destination: "drive"
  remove_obsolete: true
  sync_interval: 3600
  filters:
    # File filters to be included in syncing iCloud drive content
    folders: 
      - "Documents"
      - "Software"
    #file_extensions:
      # File extensions to be included
      #- "pdf"
      #- "png"
      #- "jpg"
      #- "jpeg"

new config:

drive:
  destination: "drive"
  remove_obsolete: true
  sync_interval: 3600
  #filters:
    # File filters to be included in syncing iCloud drive content
    #folders: 
      #- "Documents"
      #- "Software"
    #file_extensions:
      # File extensions to be included
      #- "pdf"
      #- "png"
      #- "jpg"
      #- "jpeg"
mandarons commented 1 year ago

@nicx looks like it is server side failure. I am not able to reproduce it. May be try again after some time? Also, does it work again when you switch back to old config?

nicx commented 1 year ago

@mandarons In my opinion it has nothing to do with the server side. There is indeed some download activity, I can see folders created on my local storage. Maybe there is a problem with some folders or files on iCloud Drive?! How could I get more information if this is the case?

It work immediately after I switch back to the old config. I did the change several times.

mandarons commented 1 year ago

Interesting. Please change log level to debug in your config like below:

app:
  logger:
    # level - debug, info (default), warning or error
    level: "debug"

and share the logs.

nicx commented 1 year ago

@mandarons here is the "interesting" part of the log :)

text  error  warn  system  array  login  

Traceback (most recent call last):
  File "/app/./src/main.py", line 7, in <module>
    sync.sync()
  File "/app/src/sync.py", line 72, in sync
    sync_drive.sync_drive(config=config, drive=api.drive)
  File "/app/src/sync_drive.py", line 292, in sync_drive
    return sync_directory(
  File "/app/src/sync_drive.py", line 259, in sync_directory
    sync_directory(
  File "/app/src/sync_drive.py", line 262, in sync_directory
    items=item.dir(),
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 323, in dir
    return [child.name for child in self.get_children()]
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 265, in get_children
    self.data.update(self.connection.get_node_data(self.data["drivewsid"]))
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 34, in get_node_data
    request = self.session.post(
  File "/app/venv/lib/python3.10/site-packages/requests/sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/icloudpy/base.py", line 158, in request
    self._raise_error(code, reason)
  File "/app/venv/lib/python3.10/site-packages/icloudpy/base.py", line 187, in _raise_error
    raise api_error
icloudpy.exceptions.ICloudPyAPIResponseException: Bad Request (400)
2023-01-18 07:33:37,885 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:37,885 :: DEBUG :: root :: sync_drive.py :: 132 :: File /app/icloud/drive/Downloads/Technische-Daten-Altherma-3-wandgeraet_1.svg does not exist locally.
2023-01-18 07:33:37,886 :: INFO :: root :: sync_drive.py :: 176 :: Downloading /app/icloud/drive/Downloads/Technische-Daten-Altherma-3-wandgeraet_1.svg ...
2023-01-18 07:33:37,886 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://p58-docws.icloud.com:443/ws/com.apple.CloudDocs/download/by_id 
2023-01-18 07:33:38,215 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://p58-docws.icloud.com:443 "GET /ws/com.apple.CloudDocs/download/by_id?document_id=7A298CD9-71C5-4EF8-9EBF-02A2C36514AA HTTP/1.1" 200 None
2023-01-18 07:33:38,216 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:38,217 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:38,217 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 142 :: {'document_id': '7A298CD9-71C5-4EF8-9EBF-02A2C36514AA', 'owner_dsid': 283032248, 'data_token': {'url': 'https://cvws.icloud-content.com/B/AR2bDPo1a4ZoceVZpkHplxRshHxBAZFzJdlmxZg0X7g_DdPjJmeiDTNY/Technische-Daten-Altherma-3-wandgeraet_1.svg?o=AhudVY79qs41OOHZ-4cl1UdAwWxX-kJlsBfUfYCKSfK4&v=1&x=3&a=CAoggOf-YoSW-KMCIoZtBVPawb_ozbA67V1MyKxpyOrPZggSaxDbjNWc3DAY2-mwntwwIgEAUgRshHxBWgSiDTNYaiUk0Rd-uUfayWDpFsvfKL_Gh0n1cot6Aon5ircvmGOavm9w9rUQciXTRVT9yGGs_aKm6cSHEvSTJt-ER7P1DSX2Tym1Z0CG_Crfphun&e=1674027218&fl=&r=5e8b235c-7f39-4c5c-99d2-c3bec7044d03-1&k=aoaHAuNDDyqmf-kPFfEZhg&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=UG49ENC3ONIq1SQI8TIOB3TDJWc', 'token': 'CAogbYsd2Ozdb0/vTcZJnFdrCqekEmYMFvebBfkGzQoc8vgSaxDcjNWc3DAY3OmwntwwIgEAUgRshHxBWgSiDTNYaiW6hgc8w9n4AjjOZQEXWLdhK6AR3tj/w9E6b8u6D6PCieoOJuoAciVvgmsvPwvUa6/Jb0lr5DcwAxzUdMHtlqb5xIddJAf1v8MCq0Vc', 'signature': 'AR2bDPo1a4ZoceVZpkHplxRshHxB', 'wrapping_key': 'aoaHAuNDDyqmf+kPFfEZhg==', 'reference_signature': 'AZFzJdlmxZg0X7g/DdPjJmeiDTNY'}, 'double_etag': 'tv8::tv7'}
2023-01-18 07:33:38,218 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://cvws.icloud-content.com/B/AR2bDPo1a4ZoceVZpkHplxRshHxBAZFzJdlmxZg0X7g_DdPjJmeiDTNY/Technische-Daten-Altherma-3-wandgeraet_1.svg?o=AhudVY79qs41OOHZ-4cl1UdAwWxX-kJlsBfUfYCKSfK4&v=1&x=3&a=CAoggOf-YoSW-KMCIoZtBVPawb_ozbA67V1MyKxpyOrPZggSaxDbjNWc3DAY2-mwntwwIgEAUgRshHxBWgSiDTNYaiUk0Rd-uUfayWDpFsvfKL_Gh0n1cot6Aon5ircvmGOavm9w9rUQciXTRVT9yGGs_aKm6cSHEvSTJt-ER7P1DSX2Tym1Z0CG_Crfphun&e=1674027218&fl=&r=5e8b235c-7f39-4c5c-99d2-c3bec7044d03-1&k=aoaHAuNDDyqmf-kPFfEZhg&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=UG49ENC3ONIq1SQI8TIOB3TDJWc 
2023-01-18 07:33:38,219 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 273 :: Resetting dropped connection: cvws.icloud-content.com
2023-01-18 07:33:38,470 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://cvws.icloud-content.com:443 "GET /B/AR2bDPo1a4ZoceVZpkHplxRshHxBAZFzJdlmxZg0X7g_DdPjJmeiDTNY/Technische-Daten-Altherma-3-wandgeraet_1.svg?o=AhudVY79qs41OOHZ-4cl1UdAwWxX-kJlsBfUfYCKSfK4&v=1&x=3&a=CAoggOf-YoSW-KMCIoZtBVPawb_ozbA67V1MyKxpyOrPZggSaxDbjNWc3DAY2-mwntwwIgEAUgRshHxBWgSiDTNYaiUk0Rd-uUfayWDpFsvfKL_Gh0n1cot6Aon5ircvmGOavm9w9rUQciXTRVT9yGGs_aKm6cSHEvSTJt-ER7P1DSX2Tym1Z0CG_Crfphun&e=1674027218&fl=&r=5e8b235c-7f39-4c5c-99d2-c3bec7044d03-1&k=aoaHAuNDDyqmf-kPFfEZhg&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=UG49ENC3ONIq1SQI8TIOB3TDJWc HTTP/1.1" 200 53925
2023-01-18 07:33:38,471 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:38,472 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:38,480 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://p58-docws.icloud.com:443/ws/com.apple.CloudDocs/download/by_id 
2023-01-18 07:33:38,800 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://p58-docws.icloud.com:443 "GET /ws/com.apple.CloudDocs/download/by_id?document_id=91729F1B-C3DA-4ABA-94DB-D6A525999A21 HTTP/1.1" 200 None
2023-01-18 07:33:38,801 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:38,803 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:38,803 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 142 :: {'document_id': '91729F1B-C3DA-4ABA-94DB-D6A525999A21', 'owner_dsid': 283032248, 'data_token': {'url': 'https://cvws.icloud-content.com/B/AX5smL0_h746eBoGHUxm9XLVlkg9AcRNwwAaEr9gQmlvGup7fRD9K8ub/Vertragdetails+Mobilfunk++Telekom.pdf?o=AoRKDXVZw6O4cAPAELAUAQsBmaLrgHd6xrExO1CJAoYo&v=1&x=3&a=CAogBsjKUFzPWi0RpxeKxq_KAp9F1odCZbhb4vdPO2hhdDUSaxCmkdWc3DAYpu6wntwwIgEAUgTVlkg9WgT9K8ubaiW6Gi-HjdoS3nPmxFqiZSwlPQVZ41xKuWYaf28hNOEyOe_i8PhNciUuc14TEEhQf-DtZPk0WbRhksYqBOL3do8sMhzkr_PGOP62hxsH&e=1674027218&fl=&r=68b7f7a9-3f8c-424e-8516-71ac6b3b8999-1&k=FoZXKKnrG7Kx51h2ESdnEw&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=Pemewi4THjznn0b9Ekthykze2vo', 'token': 'CAogmESOyvE9iC+b4qtzS9tUpoWnaa+Cvn5GPxr7myseNPASaxCnkdWc3DAYp+6wntwwIgEAUgTVlkg9WgT9K8ubaiWFzVWFXX4DbOUM+j2kv+U8ravCsBNDpztSsHxTVFaBQ75oLW2bciUjoJlCN3EBoTQr4j+6XfZI4r5g+fYJrHqKuzSXqoPCwXoA5No/', 'signature': 'AX5smL0/h746eBoGHUxm9XLVlkg9', 'wrapping_key': 'FoZXKKnrG7Kx51h2ESdnEw==', 'reference_signature': 'AcRNwwAaEr9gQmlvGup7fRD9K8ub'}, 'double_etag': 'tv4::tv3'}
2023-01-18 07:33:38,803 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://cvws.icloud-content.com/B/AX5smL0_h746eBoGHUxm9XLVlkg9AcRNwwAaEr9gQmlvGup7fRD9K8ub/Vertragdetails+Mobilfunk++Telekom.pdf?o=AoRKDXVZw6O4cAPAELAUAQsBmaLrgHd6xrExO1CJAoYo&v=1&x=3&a=CAogBsjKUFzPWi0RpxeKxq_KAp9F1odCZbhb4vdPO2hhdDUSaxCmkdWc3DAYpu6wntwwIgEAUgTVlkg9WgT9K8ubaiW6Gi-HjdoS3nPmxFqiZSwlPQVZ41xKuWYaf28hNOEyOe_i8PhNciUuc14TEEhQf-DtZPk0WbRhksYqBOL3do8sMhzkr_PGOP62hxsH&e=1674027218&fl=&r=68b7f7a9-3f8c-424e-8516-71ac6b3b8999-1&k=FoZXKKnrG7Kx51h2ESdnEw&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=Pemewi4THjznn0b9Ekthykze2vo 
2023-01-18 07:33:39,055 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://cvws.icloud-content.com:443 "GET /B/AX5smL0_h746eBoGHUxm9XLVlkg9AcRNwwAaEr9gQmlvGup7fRD9K8ub/Vertragdetails+Mobilfunk++Telekom.pdf?o=AoRKDXVZw6O4cAPAELAUAQsBmaLrgHd6xrExO1CJAoYo&v=1&x=3&a=CAogBsjKUFzPWi0RpxeKxq_KAp9F1odCZbhb4vdPO2hhdDUSaxCmkdWc3DAYpu6wntwwIgEAUgTVlkg9WgT9K8ubaiW6Gi-HjdoS3nPmxFqiZSwlPQVZ41xKuWYaf28hNOEyOe_i8PhNciUuc14TEEhQf-DtZPk0WbRhksYqBOL3do8sMhzkr_PGOP62hxsH&e=1674027218&fl=&r=68b7f7a9-3f8c-424e-8516-71ac6b3b8999-1&k=FoZXKKnrG7Kx51h2ESdnEw&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=Pemewi4THjznn0b9Ekthykze2vo HTTP/1.1" 200 149333
2023-01-18 07:33:39,056 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:39,058 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:39,058 :: DEBUG :: root :: sync_drive.py :: 132 :: File /app/icloud/drive/Downloads/Vertragdetails Mobilfunk  Telekom.pdf does not exist locally.
2023-01-18 07:33:39,058 :: INFO :: root :: sync_drive.py :: 176 :: Downloading /app/icloud/drive/Downloads/Vertragdetails Mobilfunk  Telekom.pdf ...
2023-01-18 07:33:39,059 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://p58-docws.icloud.com:443/ws/com.apple.CloudDocs/download/by_id 
2023-01-18 07:33:39,379 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://p58-docws.icloud.com:443 "GET /ws/com.apple.CloudDocs/download/by_id?document_id=91729F1B-C3DA-4ABA-94DB-D6A525999A21 HTTP/1.1" 200 None
2023-01-18 07:33:39,381 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:39,382 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:39,382 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 142 :: {'document_id': '91729F1B-C3DA-4ABA-94DB-D6A525999A21', 'owner_dsid': 283032248, 'data_token': {'url': 'https://cvws.icloud-content.com/B/AX5smL0_h746eBoGHUxm9XLVlkg9AcRNwwAaEr9gQmlvGup7fRD9K8ub/Vertragdetails+Mobilfunk++Telekom.pdf?o=AuZyBZJNegvI_dnwi1UPChLBqKQJvhA7wLVm8MR887yX&v=1&x=3&a=CAogXCG7xsWVc2QZLMZNheR0pulvtNZUAavfNGFxyVOOPUwSaxDoldWc3DAY6PKwntwwIgEAUgTVlkg9WgT9K8ubaiUqlkpMGpD8VczuuUFPCWG16pqGK_GCxBu6jnqsZjNkkqWDivGNciXEK4ucLegnjoseV8ZoAQXX--BOr1irpq1wEWrPNkWD8-BHTJZO&e=1674027219&fl=&r=ffc59c04-2b57-4778-9b9c-b308a4e8f57d-1&k=FoZXKKnrG7Kx51h2ESdnEw&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=-F3h2g2Q-4Ev7cPTjeeDNIAvWV8', 'token': 'CAog4Oh6bupqtSdgJs4SW9MZXTt2zUDK6dyuyi+0LFwdqasSaxDpldWc3DAY6fKwntwwIgEAUgTVlkg9WgT9K8ubaiW34f/fdPKKCnxT6hfWFuHKKfHontKPyCE0+e1BBz9W2K0YQRxlciUaF3UXqVQwQhQCWm+MQKyd3B4kUriKv+kcSFWGvIL16vovRboj', 'signature': 'AX5smL0/h746eBoGHUxm9XLVlkg9', 'wrapping_key': 'FoZXKKnrG7Kx51h2ESdnEw==', 'reference_signature': 'AcRNwwAaEr9gQmlvGup7fRD9K8ub'}, 'double_etag': 'tv4::tv3'}
2023-01-18 07:33:39,383 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://cvws.icloud-content.com/B/AX5smL0_h746eBoGHUxm9XLVlkg9AcRNwwAaEr9gQmlvGup7fRD9K8ub/Vertragdetails+Mobilfunk++Telekom.pdf?o=AuZyBZJNegvI_dnwi1UPChLBqKQJvhA7wLVm8MR887yX&v=1&x=3&a=CAogXCG7xsWVc2QZLMZNheR0pulvtNZUAavfNGFxyVOOPUwSaxDoldWc3DAY6PKwntwwIgEAUgTVlkg9WgT9K8ubaiUqlkpMGpD8VczuuUFPCWG16pqGK_GCxBu6jnqsZjNkkqWDivGNciXEK4ucLegnjoseV8ZoAQXX--BOr1irpq1wEWrPNkWD8-BHTJZO&e=1674027219&fl=&r=ffc59c04-2b57-4778-9b9c-b308a4e8f57d-1&k=FoZXKKnrG7Kx51h2ESdnEw&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=-F3h2g2Q-4Ev7cPTjeeDNIAvWV8 
2023-01-18 07:33:39,384 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 273 :: Resetting dropped connection: cvws.icloud-content.com
2023-01-18 07:33:39,637 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://cvws.icloud-content.com:443 "GET /B/AX5smL0_h746eBoGHUxm9XLVlkg9AcRNwwAaEr9gQmlvGup7fRD9K8ub/Vertragdetails+Mobilfunk++Telekom.pdf?o=AuZyBZJNegvI_dnwi1UPChLBqKQJvhA7wLVm8MR887yX&v=1&x=3&a=CAogXCG7xsWVc2QZLMZNheR0pulvtNZUAavfNGFxyVOOPUwSaxDoldWc3DAY6PKwntwwIgEAUgTVlkg9WgT9K8ubaiUqlkpMGpD8VczuuUFPCWG16pqGK_GCxBu6jnqsZjNkkqWDivGNciXEK4ucLegnjoseV8ZoAQXX--BOr1irpq1wEWrPNkWD8-BHTJZO&e=1674027219&fl=&r=ffc59c04-2b57-4778-9b9c-b308a4e8f57d-1&k=FoZXKKnrG7Kx51h2ESdnEw&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=-F3h2g2Q-4Ev7cPTjeeDNIAvWV8 HTTP/1.1" 200 149333
2023-01-18 07:33:39,638 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:39,639 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:39,660 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://p58-docws.icloud.com:443/ws/com.apple.CloudDocs/download/by_id 
2023-01-18 07:33:39,950 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://p58-docws.icloud.com:443 "GET /ws/com.apple.CloudDocs/download/by_id?document_id=33564AFC-96BC-43D3-8427-EE5A65469CFF HTTP/1.1" 200 None
2023-01-18 07:33:39,951 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:39,953 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:39,954 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 142 :: {'document_id': '33564AFC-96BC-43D3-8427-EE5A65469CFF', 'owner_dsid': 283032248, 'data_token': {'url': 'https://cvws.icloud-content.com/B/AfJXnyi-PhJ-Z7_iZs6SaYvDoSjoAW5zy6AuszC7ZQySs9RHtSRDRlYn/Zubehoer-Altherma-3-Wandgeraet_1.svg?o=Alh8kKNIPkK7SdL8LsE_TRCOk3mMI3umMcs4GBX15WPz&v=1&x=3&a=CAogA8CRqI3kB6hIkxo-0koE8JKROvrmC1inmDc5Q0UTAQ8SaxCkmtWc3DAYpPewntwwIgEAUgTDoSjoWgRDRlYnaiWpvT3uZGLbs8XWq0SC5aA18VaAqPMkGh7Uhnv7jnTJzmFWi9adciWwWjSbuEyzylX-K7Cks2737gCAuRrrRH6o2PYxhsucxr560Ot5&e=1674027219&fl=&r=839c0c75-f40c-4c00-94e8-bc370ad3808e-1&k=hU6xYENNPwLJ5SzczrzhZA&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=iJ8PWguO3A4WhbNg6XWhTqQ1QTc', 'token': 'CAogidCiu8gqtzENQpqvEUhtt4SPC4rpHeGyydjWja3d8w8SaxClmtWc3DAYpfewntwwIgEAUgTDoSjoWgRDRlYnaiWTch37kbZnjRfPc2/hqsfPE0/x6zsiUOiqcCIo9ACMhP0tp/uRciV5HsAFtibi3+t+j6y9XCqlItKUO0QdBAS8tIiYx8HfR/gpoGm5', 'signature': 'AfJXnyi+PhJ+Z7/iZs6SaYvDoSjo', 'wrapping_key': 'hU6xYENNPwLJ5SzczrzhZA==', 'reference_signature': 'AW5zy6AuszC7ZQySs9RHtSRDRlYn'}, 'double_etag': 'tvc::tvb'}
2023-01-18 07:33:39,954 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://cvws.icloud-content.com/B/AfJXnyi-PhJ-Z7_iZs6SaYvDoSjoAW5zy6AuszC7ZQySs9RHtSRDRlYn/Zubehoer-Altherma-3-Wandgeraet_1.svg?o=Alh8kKNIPkK7SdL8LsE_TRCOk3mMI3umMcs4GBX15WPz&v=1&x=3&a=CAogA8CRqI3kB6hIkxo-0koE8JKROvrmC1inmDc5Q0UTAQ8SaxCkmtWc3DAYpPewntwwIgEAUgTDoSjoWgRDRlYnaiWpvT3uZGLbs8XWq0SC5aA18VaAqPMkGh7Uhnv7jnTJzmFWi9adciWwWjSbuEyzylX-K7Cks2737gCAuRrrRH6o2PYxhsucxr560Ot5&e=1674027219&fl=&r=839c0c75-f40c-4c00-94e8-bc370ad3808e-1&k=hU6xYENNPwLJ5SzczrzhZA&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=iJ8PWguO3A4WhbNg6XWhTqQ1QTc 
2023-01-18 07:33:40,202 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://cvws.icloud-content.com:443 "GET /B/AfJXnyi-PhJ-Z7_iZs6SaYvDoSjoAW5zy6AuszC7ZQySs9RHtSRDRlYn/Zubehoer-Altherma-3-Wandgeraet_1.svg?o=Alh8kKNIPkK7SdL8LsE_TRCOk3mMI3umMcs4GBX15WPz&v=1&x=3&a=CAogA8CRqI3kB6hIkxo-0koE8JKROvrmC1inmDc5Q0UTAQ8SaxCkmtWc3DAYpPewntwwIgEAUgTDoSjoWgRDRlYnaiWpvT3uZGLbs8XWq0SC5aA18VaAqPMkGh7Uhnv7jnTJzmFWi9adciWwWjSbuEyzylX-K7Cks2737gCAuRrrRH6o2PYxhsucxr560Ot5&e=1674027219&fl=&r=839c0c75-f40c-4c00-94e8-bc370ad3808e-1&k=hU6xYENNPwLJ5SzczrzhZA&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=iJ8PWguO3A4WhbNg6XWhTqQ1QTc HTTP/1.1" 200 759815
2023-01-18 07:33:40,203 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:40,205 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:40,205 :: DEBUG :: root :: sync_drive.py :: 132 :: File /app/icloud/drive/Downloads/Zubehoer-Altherma-3-Wandgeraet_1.svg does not exist locally.
2023-01-18 07:33:40,205 :: INFO :: root :: sync_drive.py :: 176 :: Downloading /app/icloud/drive/Downloads/Zubehoer-Altherma-3-Wandgeraet_1.svg ...
2023-01-18 07:33:40,206 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://p58-docws.icloud.com:443/ws/com.apple.CloudDocs/download/by_id 
2023-01-18 07:33:40,515 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://p58-docws.icloud.com:443 "GET /ws/com.apple.CloudDocs/download/by_id?document_id=33564AFC-96BC-43D3-8427-EE5A65469CFF HTTP/1.1" 200 None
2023-01-18 07:33:40,516 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:40,518 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:40,518 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 142 :: {'document_id': '33564AFC-96BC-43D3-8427-EE5A65469CFF', 'owner_dsid': 283032248, 'data_token': {'url': 'https://cvws.icloud-content.com/B/AfJXnyi-PhJ-Z7_iZs6SaYvDoSjoAW5zy6AuszC7ZQySs9RHtSRDRlYn/Zubehoer-Altherma-3-Wandgeraet_1.svg?o=AmgpLCYiPiq40mY6jsYVpEF7B7SqfJp8SODa3qKaP9hF&v=1&x=3&a=CAogJYzCivXVXdI3A-eVE1pE9D3c5ecdPg9tgGeT9RkgurUSaxDEntWc3DAYxPuwntwwIgEAUgTDoSjoWgRDRlYnaiWp1tU11AkZLMNGf5nUi7Q7gEpms5CURcFjR-ddRAuIzMH95XsociW2P9TlCtCwofwT0CKZRe9D2DlHfamaaEUetp2NHS0zZMnFV5FH&e=1674027220&fl=&r=5d75002f-7b34-40d8-af8b-eda263c37a63-1&k=hU6xYENNPwLJ5SzczrzhZA&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=qViVA4EquelHuNrYa2xhGq4jo-E', 'token': 'CAogdK4melwR90qbi0hThAGb66AdeeYZvRc8eTYkZho/4ygSaxDEntWc3DAYxPuwntwwIgEAUgTDoSjoWgRDRlYnaiUoOcQ+IzRtlXbs6v55eGFOfoSf2SQbbo++8du4G998zFa2nSeJciWQq2U8GcQg2ELsKta7Nqykx1ejKZWNMs6F4uh1YegKHMc5UnfG', 'signature': 'AfJXnyi+PhJ+Z7/iZs6SaYvDoSjo', 'wrapping_key': 'hU6xYENNPwLJ5SzczrzhZA==', 'reference_signature': 'AW5zy6AuszC7ZQySs9RHtSRDRlYn'}, 'double_etag': 'tvc::tvb'}
2023-01-18 07:33:40,519 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: GET https://cvws.icloud-content.com/B/AfJXnyi-PhJ-Z7_iZs6SaYvDoSjoAW5zy6AuszC7ZQySs9RHtSRDRlYn/Zubehoer-Altherma-3-Wandgeraet_1.svg?o=AmgpLCYiPiq40mY6jsYVpEF7B7SqfJp8SODa3qKaP9hF&v=1&x=3&a=CAogJYzCivXVXdI3A-eVE1pE9D3c5ecdPg9tgGeT9RkgurUSaxDEntWc3DAYxPuwntwwIgEAUgTDoSjoWgRDRlYnaiWp1tU11AkZLMNGf5nUi7Q7gEpms5CURcFjR-ddRAuIzMH95XsociW2P9TlCtCwofwT0CKZRe9D2DlHfamaaEUetp2NHS0zZMnFV5FH&e=1674027220&fl=&r=5d75002f-7b34-40d8-af8b-eda263c37a63-1&k=hU6xYENNPwLJ5SzczrzhZA&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=qViVA4EquelHuNrYa2xhGq4jo-E 
2023-01-18 07:33:40,521 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 273 :: Resetting dropped connection: cvws.icloud-content.com
2023-01-18 07:33:40,848 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://cvws.icloud-content.com:443 "GET /B/AfJXnyi-PhJ-Z7_iZs6SaYvDoSjoAW5zy6AuszC7ZQySs9RHtSRDRlYn/Zubehoer-Altherma-3-Wandgeraet_1.svg?o=AmgpLCYiPiq40mY6jsYVpEF7B7SqfJp8SODa3qKaP9hF&v=1&x=3&a=CAogJYzCivXVXdI3A-eVE1pE9D3c5ecdPg9tgGeT9RkgurUSaxDEntWc3DAYxPuwntwwIgEAUgTDoSjoWgRDRlYnaiWp1tU11AkZLMNGf5nUi7Q7gEpms5CURcFjR-ddRAuIzMH95XsociW2P9TlCtCwofwT0CKZRe9D2DlHfamaaEUetp2NHS0zZMnFV5FH&e=1674027220&fl=&r=5d75002f-7b34-40d8-af8b-eda263c37a63-1&k=hU6xYENNPwLJ5SzczrzhZA&ckc=com.apple.clouddocs&ckz=com.apple.CloudDocs&p=58&s=qViVA4EquelHuNrYa2xhGq4jo-E HTTP/1.1" 200 759815
2023-01-18 07:33:40,849 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:40,850 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:40,879 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: POST https://p58-drivews.icloud.com:443/retrieveItemDetailsInFolders [{"drivewsid": "SHARED_FOLDER::com.apple.CloudDocs::86FE387A-64D4-3FDF-8C6A-7E9E6A5E8FFB", "partialData": false}]
2023-01-18 07:33:41,504 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://p58-drivews.icloud.com:443 "POST /retrieveItemDetailsInFolders HTTP/1.1" 200 None
2023-01-18 07:33:41,506 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:41,509 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:41,509 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 142 :: [{'dateCreated': '2022-09-21T07:44:23Z', 'drivewsid': 'SHARED_FOLDER::com.apple.CloudDocs::86FE387A-64D4-3FDF-8C6A-7E9E6A5E8FFB', 'docwsid': '86FE387A-64D4-3FDF-8C6A-7E9E6A5E8FFB', 'zone': 'com.apple.CloudDocs', 'name': 'Familie_Dokumente', 'parentId': 'FOLDER::com.apple.CloudDocs::root', 'etag': 'qnn::3v', 'type': 'FOLDER', 'shortGUID': '097H9ZdmhQN7A9gyH7ty-Joag', 'shareID': {'shareName': '5E34AAD9-8015-4899-AF95-2B640B27ACF3', 'recordName': '5E34AAD9-8015-4899-AF95-2B640B27ACF3', 'zoneID': {'zoneName': 'com.apple.CloudDocs', 'ownerRecordName': '_488207b328a50109f4bfb1845ed3ba5f'}}, 'share': 'TO_ME', 'assetQuota': 8114740, 'fileCount': 1, 'shareCount': 1, 'shareAliasCount': 0, 'directChildrenCount': 2, 'owner': {'firstName': 'Familie', 'lastName': 'Klinger', 'ownerRecordName': '_488207b328a50109f4bfb1845ed3ba5f'}, 'items': [{'dateCreated': '2022-09-23T13:11:42Z', 'drivewsid': 'FOLDER_IN_SHARED_FOLDER::com.apple.CloudDocs::C7DFCC79-B8F3-4CEF-B108-0FF35778E49C', 'docwsid': 'C7DFCC79-B8F3-4CEF-B108-0FF35778E49C', 'zone': 'com.apple.CloudDocs', 'name': 'max', 'parentId': 'SHARED_FOLDER::com.apple.CloudDocs::86FE387A-64D4-3FDF-8C6A-7E9E6A5E8FFB', 'isChainedToParent': True, 'etag': '38::', 'type': 'FOLDER', 'shareID': {'shareName': '5E34AAD9-8015-4899-AF95-2B640B27ACF3', 'recordName': '5E34AAD9-8015-4899-AF95-2B640B27ACF3', 'zoneID': {'zoneName': 'com.apple.CloudDocs', 'ownerRecordName': '_488207b328a50109f4bfb1845ed3ba5f'}}, 'share': 'TO_ME', 'assetQuota': 0, 'fileCount': 0, 'shareCount': 0, 'shareAliasCount': 0, 'directChildrenCount': 0, 'owner': {'firstName': 'Familie', 'lastName': 'Klinger', 'ownerRecordName': '_488207b328a50109f4bfb1845ed3ba5f'}}, {'dateCreated': '2022-04-29T14:46:39Z', 'drivewsid': 'FILE_IN_SHARED_FOLDER::com.apple.CloudDocs::30C8E5B8-224A-4C90-8B6F-CBDA39289BDF', 'docwsid': '30C8E5B8-224A-4C90-8B6F-CBDA39289BDF', 'zone': 'com.apple.CloudDocs', 'name': 'SUN2000-(29.9KTL, 33KTL-A, 36KTL, 42KTL) Benutzerhandbuch', 'extension': 'pdf', 'parentId': 'SHARED_FOLDER::com.apple.CloudDocs::86FE387A-64D4-3FDF-8C6A-7E9E6A5E8FFB', 'isChainedToParent': True, 'dateModified': '2022-04-29T14:46:39Z', 'dateChanged': '2022-09-21T07:50:37Z', 'size': 8114740, 'etag': '2y::31', 'share': 'TO_ME', 'shareID': {'shareName': '5E34AAD9-8015-4899-AF95-2B640B27ACF3', 'recordName': '5E34AAD9-8015-4899-AF95-2B640B27ACF3', 'zoneID': {'zoneName': 'com.apple.CloudDocs', 'ownerRecordName': '_488207b328a50109f4bfb1845ed3ba5f'}}, 'shortGUID': '0cfNiOzrjE3NPcbkKK8DmLa0Q', 'lastOpenTime': '2022-09-23T13:11:38Z', 'unifiedToken': 'AwC2CAESEIM4CF81vq3537RTzVRe-mQacgMjNU9zicC7ExcB1AJ-SvTfdCwNLYg9ghPWxopiQq6GTYjYUmNXpVDXPozi146QtZOvGFgdXunPTBY9ni9SkwfQKKJwk2yObhv8LxN5pZQGxvG11-nS72enTsHiwGszjStNoWbEoo3Nm_tv61cjwuzPegCAESkgF7gyMCTol8B9cHrMkHKSzxL3JOmW6v0ZxuTOFV0fmZYQa-TwdwxwUruI0mQtFdGaEA0eOYI6ZLkm8dpr2owrn1o-9ZX3yf0oiv9fvUripPyTJ3Z6ZFpKQzgxOM2Xz51R9Tzo8FTOrV03nVYunC-bB90Spy_M_Q2m_moy7Yzy1YJpEnALLf6NNB6Fa-mRUD2ybmhRgBIhCoqpTMfQ0bFmjx1Mz0zPHbKiQ2NzUzYTg5Ni1lYjQ2LTRhMzctYmQyNS0yODM4MDQ4YjY5N2E', 'type': 'FILE', 'owner': {'firstName': 'Familie', 'lastName': 'Klinger', 'ownerRecordName': '_488207b328a50109f4bfb1845ed3ba5f'}}], 'numberOfItems': 2, 'status': 'OK'}]
2023-01-18 07:33:41,512 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 75 :: POST https://p58-drivews.icloud.com:443/retrieveItemDetailsInFolders [{"drivewsid": "FOLDER_IN_SHARED_FOLDER::com.apple.CloudDocs::C7DFCC79-B8F3-4CEF-B108-0FF35778E49C", "partialData": false}]
2023-01-18 07:33:41,691 :: DEBUG :: urllib3.connectionpool :: connectionpool.py :: 456 :: https://p58-drivews.icloud.com:443 "POST /retrieveItemDetailsInFolders HTTP/1.1" 400 None
2023-01-18 07:33:41,694 :: DEBUG :: icloudpy.base :: base.py :: 94 :: Saved session data to file
2023-01-18 07:33:41,697 :: DEBUG :: icloudpy.base :: base.py :: 98 :: Cookies saved to session_data/**********de
2023-01-18 07:33:41,697 :: DEBUG :: icloudpy.services.drive.http :: base.py :: 142 :: {'requestUUID': '31791bf4-8903-4611-9c10-aa565ad1683a', 'errorReason': 'Bad Request', 'errorCode': 400}
2023-01-18 07:33:41,697 :: ERROR :: icloudpy.base :: base.py :: 186 :: Bad Request (400)

** Press ANY KEY to close this window **
mandarons commented 1 year ago

It looks like failure is in listing the Familie_Dokumente directory for some reason. It looks like it has two items in it. Anything unusual about it - e.g. special characters, hidden etc?

nicx commented 1 year ago

@mandarons it is a file called "SUN2000-(29.9KTL, 33KTL-A, 36KTL, 42KTL) Benutzerhandbuch.pdf". So you are right, its name contain special characters. I think iCloud docker should deal with that, so if it cannot download the file it should throw a warning, but it should not just crash ;)

mandarons commented 1 year ago

I implemented a workaround for this error. However, the root cause is in icloudpy - problem is it doesn't handle shared folders (i.e. folders shared by others with the user). The workaround should allow execution to continue.

mandarons commented 1 year ago

https://github.com/mandarons/icloudpy/issues/12 created to track shared folder support.