search

Duke-GCB / DukeDSClient

Command line program to allow uploading, downloading, and managing projects in the duke-data-service.
MIT License
5 stars 6 forks source link

ConnectionResetError downloading large files on windows #282

Closed johnbradley closed 4 years ago

johnbradley commented 4 years ago

A user on a windows machine encountered the following error when downloading large files:

Traceback (most recent call last):
  File "c:\program files\python\lib\site-packages\ddsc\core\download.py", line 417, in run
    return self.retry_download_loop()
  File "c:\program files\python\lib\site-packages\ddsc\core\download.py", line 430, in retry_download_loop
    self.download_chunk(url, headers)
  File "c:\program files\python\lib\site-packages\ddsc\core\download.py", line 472, in download_chunk
    self._write_response_to_file(response)
  File "c:\program files\python\lib\site-packages\ddsc\core\download.py", line 482, in _write_response_to_file
    for chunk in response.iter_content(chunk_size=self.bytes_per_chunk):
...
requests.exceptions.ChunkedEncodingError: ("Connection broken: 
    ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", 
    ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

On another windows machine this error was not a problem for downloading larger files.

johnbradley commented 4 years ago

After setting the download_workers ddsclient config option to 1 the user was able to download the project without error. The problem seems to be related to issuing multiple get requests to the same URL with different range requests.