search

icloud-photos-downloader / icloud_photos_downloader

A command-line tool to download photos from iCloud
MIT License
6.57k stars 534 forks source link

Unhandled exception: http.client.RemoteDisconnected: Remote end closed connection without response #711

Closed Sevyls closed 9 months ago

Sevyls commented 10 months ago

Overview

it stops suddenly due to an unhandled connection exception: "http.client.RemoteDisconnected: Remote end closed connection without response"

Steps to Reproduce

  1. Run long running download of whole library (all photos, in my case over 120k items),
  2. leave it running for 15 hours as it isnt finished yet

Expected Behavior

error log entry, a correct Exception handling of this connection issue. it should then continue download after a reconnect

Actual Behavior

it stopped downloading and exited with a unhandled exception and stacktrace.

Context

// 15 hours of correct download....
2023-11-06 01:29:39 INFO     Downloaded D:\iCloud-Fotos\2014\09\21\2014-09-21 16.18.25.jpg
Traceback (most recent call last):
  File "urllib3\connectionpool.py", line 714, in urlopen
  File "urllib3\connectionpool.py", line 466, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.py", line 461, in _make_request
  File "http\client.py", line 1378, in getresponse
  File "http\client.py", line 318, in begin
  File "http\client.py", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests\adapters.py", line 486, in send
  File "urllib3\connectionpool.py", line 798, in urlopen
  File "urllib3\util\retry.py", line 550, in increment
  File "urllib3\packages\six.py", line 769, in reraise
  File "urllib3\connectionpool.py", line 714, in urlopen
  File "urllib3\connectionpool.py", line 466, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.py", line 461, in _make_request
  File "http\client.py", line 1378, in getresponse
  File "http\client.py", line 318, in begin
  File "http\client.py", line 287, in _read_status
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "starters\icloudpd.py", line 5, in <module>
  File "click\core.py", line 1157, in __call__
  File "click\core.py", line 1078, in main
  File "click\core.py", line 1434, in invoke
  File "click\core.py", line 783, in invoke
  File "icloudpd\base.py", line 317, in main
  File "icloudpd\base.py", line 889, in core
  File "tqdm\std.py", line 1180, in __iter__
  File "pyicloud_ipd\services\photos.py", line 357, in photos
  File "pyicloud_ipd\services\photos.py", line 337, in photos_request
  File "requests\sessions.py", line 637, in post
  File "pyicloud_ipd\base.py", line 71, in request
  File "requests\sessions.py", line 589, in request
  File "requests\sessions.py", line 703, in send
  File "requests\adapters.py", line 501, in send
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
[19040] Failed to execute script 'icloudpd' due to unhandled exception!
iammarxg commented 9 months ago

Same here. 44097 items to download. Stopped around 38k after ~5 hours. Then again I believe at 42k (though this time after an hour or so)?

First time:


Traceback (most recent call last):
  File "urllib3\connectionpool.py", line 714, in urlopen
  File "urllib3\connectionpool.py", line 466, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.py", line 461, in _make_request
  File "http\client.py", line 1378, in getresponse
  File "http\client.py", line 318, in begin
  File "http\client.py", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests\adapters.py", line 486, in send
  File "urllib3\connectionpool.py", line 798, in urlopen
  File "urllib3\util\retry.py", line 550, in increment
  File "urllib3\packages\six.py", line 769, in reraise
  File "urllib3\connectionpool.py", line 714, in urlopen
  File "urllib3\connectionpool.py", line 466, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.py", line 461, in _make_request
  File "http\client.py", line 1378, in getresponse
  File "http\client.py", line 318, in begin
  File "http\client.py", line 287, in _read_status
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "starters\icloudpd.py", line 5, in <module>
  File "click\core.py", line 1157, in __call__
  File "click\core.py", line 1078, in main
  File "click\core.py", line 1434, in invoke
  File "click\core.py", line 783, in invoke
  File "icloudpd\base.py", line 317, in main
  File "icloudpd\base.py", line 889, in core
  File "tqdm\std.py", line 1180, in __iter__
  File "pyicloud_ipd\services\photos.py", line 357, in photos
  File "pyicloud_ipd\services\photos.py", line 337, in photos_request
  File "requests\sessions.py", line 637, in post
  File "pyicloud_ipd\base.py", line 71, in request
  File "requests\sessions.py", line 589, in request
  File "requests\sessions.py", line 703, in send
  File "requests\adapters.py", line 501, in send
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
[23232] Failed to execute script 'icloudpd' due to unhandled exception!```

----------------------------------------------------------------------------------------
### Again:
```Traceback (most recent call last):
  File "urllib3\connectionpool.py", line 714, in urlopen
  File "urllib3\connectionpool.py", line 466, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.py", line 461, in _make_request
  File "http\client.py", line 1378, in getresponse
  File "http\client.py", line 318, in begin
  File "http\client.py", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests\adapters.py", line 486, in send
  File "urllib3\connectionpool.py", line 798, in urlopen
  File "urllib3\util\retry.py", line 550, in increment
  File "urllib3\packages\six.py", line 769, in reraise
  File "urllib3\connectionpool.py", line 714, in urlopen
  File "urllib3\connectionpool.py", line 466, in _make_request
  File "<string>", line 3, in raise_from
  File "urllib3\connectionpool.py", line 461, in _make_request
  File "http\client.py", line 1378, in getresponse
  File "http\client.py", line 318, in begin
  File "http\client.py", line 287, in _read_status
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "starters\icloudpd.py", line 5, in <module>
  File "click\core.py", line 1157, in __call__
  File "click\core.py", line 1078, in main
  File "click\core.py", line 1434, in invoke
  File "click\core.py", line 783, in invoke
  File "icloudpd\base.py", line 317, in main
  File "icloudpd\base.py", line 889, in core
  File "tqdm\std.py", line 1180, in __iter__
  File "pyicloud_ipd\services\photos.py", line 357, in photos
  File "pyicloud_ipd\services\photos.py", line 337, in photos_request
  File "requests\sessions.py", line 637, in post
  File "pyicloud_ipd\base.py", line 71, in request
  File "requests\sessions.py", line 589, in request
  File "requests\sessions.py", line 703, in send
  File "requests\adapters.py", line 501, in send
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
[32740] Failed to execute script 'icloudpd' due to unhandled exception!```
AndreyNikiforov commented 9 months ago

The error suggests that connection was dropped by Apple server. It may be result of the throttling. You can start download again and it will download only new files (will not re-download existing ones).