AlexAplin / nndownload

Download and process links from Niconico (nicovideo.jp)
MIT License
223 stars 29 forks source link

Catch exceptions generally #117

Closed Airmomi closed 2 years ago

Airmomi commented 2 years ago

When the network is interrupted, the following error is displayed.

  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 438, in _error_catcher
    yield
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 519, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/lib/python3.9/http/client.py", line 462, in read
    n = self.readinto(b)
  File "/usr/lib/python3.9/http/client.py", line 506, in readinto
    n = self.fp.readinto(b)
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/models.py", line 753, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 576, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 541, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 455, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/nndownload/nndownload.py", line 1772, in main
    process_url_mo(session, url_mo)
  File "/usr/local/lib/python3.9/dist-packages/nndownload/nndownload.py", line 1722, in process_url_mo
    request_video(session, url_id)
  File "/usr/local/lib/python3.9/dist-packages/nndownload/nndownload.py", line 982, in request_video
    download_video(session, filename, template_params)
  File "/usr/local/lib/python3.9/dist-packages/nndownload/nndownload.py", line 1251, in download_video
    for block in stream_iterator:
  File "/usr/lib/python3/dist-packages/requests/models.py", line 756, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))

In order to prevent download interruption caused by network interruption, I tried to catch these exceptions then download again ,but i just can't catch these exceptions

try:
    nndownload.execute('...')
    print('download successful')
except Exception:
    print('download failed')

the above code will print the above error and ”download successful“ , no exception is caught at all.

AlexAplin commented 2 years ago

Exceptions will now be raised fully when only one URL is passed. When multiple URLs are passed on the command line, all URLs will be parsed if they match our custom exceptions (same as with mylists, series, and more), while other exceptions will raise. In the future I may add an argument to halt downloading on any exception but for now I prefer this permissive approach.