Repeated aborted connections - forcibly closed.

[JamesLeeLab]

Repeated failed connections - aborted, forcibly closed by the remote host

initially worked fine, since 21/07/2023 8pm BST repeated aborted connections

Code, OS and python version in output below:

(base) C:\Users\leej1>pyega3 -cf "Z:\home\shared\Data\credentials_jcl.json" -c 40 -ms 1073741824 fetch EGAD00001008758 --output-dir "Z:\home\shared\Data\Ank_spond_Brown_Cell_Genomics" [2023-07-21 23:41:49 +0100] [2023-07-21 23:41:49 +0100] pyEGA3 - EGA python client version 5.0.2 (https://github.com/EGA-archive/ega-download-client) [2023-07-21 23:41:49 +0100] Parts of this software are derived from pyEGA (https://github.com/blachlylab/pyega) by James Blachly [2023-07-21 23:41:49 +0100] Python version : 3.9.17 [2023-07-21 23:41:49 +0100] OS version : Windows 10.0.19045 [2023-07-21 23:41:49 +0100] Server URL: https://ega.ebi.ac.uk:8443/v2 [2023-07-21 23:41:49 +0100] Session-Id: 2535863944 [2023-07-21 23:41:50 +0100] [2023-07-21 23:41:50 +0100] Authentication success for user 'james.lee@crick.ac.uk' ... #removed for length constraints [2023-07-22 00:05:56 +0100] File Id: 'EGAF00006161731'(2894525789 bytes). [2023-07-22 00:05:56 +0100] Total space : 476.33 GiB [2023-07-22 00:05:56 +0100] Used space : 375.51 GiB [2023-07-22 00:05:56 +0100] Free space : 100.82 GiB [2023-07-22 00:05:56 +0100] Download starting [using 40 connection(s), file size 2894525773 and chunk length 1073741824]... 0%| | 0.00/2.89G [00:00<?, ?B/s][2023-07-22 00:10:57 +0100] Retrying (Retry(total=19, connect=False, read=9, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:10:58 +0100] Retrying (Retry(total=19, connect=False, read=9, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:11:01 +0100] Retrying (Retry(total=19, connect=False, read=9, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:16:00 +0100] Retrying (Retry(total=18, connect=False, read=8, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:16:01 +0100] Retrying (Retry(total=18, connect=False, read=8, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:16:04 +0100] Retrying (Retry(total=18, connect=False, read=8, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:21:06 +0100] Retrying (Retry(total=17, connect=False, read=7, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:21:06 +0100] Retrying (Retry(total=17, connect=False, read=7, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:21:08 +0100] Retrying (Retry(total=17, connect=False, read=7, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:26:11 +0100] Retrying (Retry(total=16, connect=False, read=6, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:26:16 +0100] Retrying (Retry(total=16, connect=False, read=6, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:26:16 +0100] Retrying (Retry(total=16, connect=False, read=6, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:31:23 +0100] Retrying (Retry(total=15, connect=False, read=5, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:31:26 +0100] Retrying (Retry(total=15, connect=False, read=5, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:31:28 +0100] Retrying (Retry(total=15, connect=False, read=5, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:36:43 +0100] Retrying (Retry(total=14, connect=False, read=4, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:36:45 +0100] Retrying (Retry(total=14, connect=False, read=4, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:36:46 +0100] Retrying (Retry(total=14, connect=False, read=4, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:42:26 +0100] Retrying (Retry(total=13, connect=False, read=3, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:42:26 +0100] Retrying (Retry(total=13, connect=False, read=3, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:42:27 +0100] Retrying (Retry(total=13, connect=False, read=3, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:48:43 +0100] Retrying (Retry(total=12, connect=False, read=2, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:48:45 +0100] Retrying (Retry(total=12, connect=False, read=2, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:48:45 +0100] Retrying (Retry(total=12, connect=False, read=2, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:55:45 +0100] Retrying (Retry(total=11, connect=False, read=1, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:55:47 +0100] Retrying (Retry(total=11, connect=False, read=1, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 00:55:48 +0100] Retrying (Retry(total=11, connect=False, read=1, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:02:45 +0100] Retrying (Retry(total=10, connect=False, read=0, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:02:50 +0100] Retrying (Retry(total=10, connect=False, read=0, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:02:51 +0100] Retrying (Retry(total=10, connect=False, read=0, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain 0%| | 0.00/2.89G [1:01:57<?, ?B/s] [2023-07-22 01:07:53 +0100] HTTPSConnectionPool(host='ega.ebi.ac.uk', port=8443): Max retries exceeded with url: /v2/files/EGAF00006161731?destinationFormat=plain (Caused by ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))) Traceback (most recent call last): File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "C:\Users\leej1\Anaconda3\lib\http\client.py", line 1377, in getresponse response.begin() File "C:\Users\leej1\Anaconda3\lib\http\client.py", line 320, in begin version, status, reason = self._read_status() File "C:\Users\leej1\Anaconda3\lib\http\client.py", line 281, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "C:\Users\leej1\Anaconda3\lib\socket.py", line 704, in readinto return self._sock.recv_into(b) File "C:\Users\leej1\Anaconda3\lib\ssl.py", line 1242, in recv_into return self.read(nbytes, buffer) File "C:\Users\leej1\Anaconda3\lib\ssl.py", line 1100, in read return self._sslobj.read(len, buffer) ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\adapters.py", line 439, in send resp = conn.urlopen( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen return self.urlopen( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen return self.urlopen( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen return self.urlopen( [Previous line repeated 7 more times] File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen retries = retries.increment( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\util\retry.py", line 574, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='ega.ebi.ac.uk', port=8443): Max retries exceeded with url: /v2/files/EGAF00006161731?destinationFormat=plain (Caused by ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_file.py", line 323, in download_file_retry self.download_file(output_file, num_connections, max_slice_size) File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_file.py", line 159, in download_file for part_file_name in executor.map(self.download_fileslice, params): File "C:\Users\leej1\Anaconda3\lib\concurrent\futures_base.py", line 609, in result_iterator yield fs.pop().result() File "C:\Users\leej1\Anaconda3\lib\concurrent\futures_base.py", line 446, in result return self.get_result() File "C:\Users\leej1\Anaconda3\lib\concurrent\futures_base.py", line 391, in get_result raise self._exception File "C:\Users\leej1\Anaconda3\lib\concurrent\futures\thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_file.py", line 189, in download_fileslice return self.download_file_slice(args) File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_file.py", line 221, in download_file_slice with self.data_client.get_stream(path, extra_headers) as r: File "C:\Users\leej1\Anaconda3\lib\contextlib.py", line 119, in enter return next(self.gen) File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_client.py", line 76, in get_stream with self.session.get(url, headers=headers, stream=True, timeout=request_timeout_in_sec) as r: File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\sessions.py", line 555, in get return self.request('GET', url, kwargs) File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\sessions.py", line 542, in request resp = self.send(prep, send_kwargs) File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='ega.ebi.ac.uk', port=8443): Max retries exceeded with url: /v2/files/EGAF00006161731?destinationFormat=plain (Caused by ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))) [2023-07-22 01:08:53 +0100] retry attempt 1 [2023-07-22 01:08:53 +0100] Download starting [using 40 connection(s), file size 2894525773 and chunk length 1073741824]... 0%| | 0.00/2.89G [00:00<?, ?B/s][2023-07-22 01:08:54 +0100] [2023-07-22 01:08:54 +0100] Authentication success for user 'james.lee@crick.ac.uk' [2023-07-22 01:08:54 +0100] [2023-07-22 01:08:54 +0100] Authentication success for user 'james.lee@crick.ac.uk' [2023-07-22 01:08:54 +0100] [2023-07-22 01:08:54 +0100] Authentication success for user 'james.lee@crick.ac.uk' [2023-07-22 01:13:55 +0100] Retrying (Retry(total=19, connect=False, read=9, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:13:55 +0100] Retrying (Retry(total=19, connect=False, read=9, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:13:57 +0100] Retrying (Retry(total=19, connect=False, read=9, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:18:57 +0100] Retrying (Retry(total=18, connect=False, read=8, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:18:57 +0100] Retrying (Retry(total=18, connect=False, read=8, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:18:58 +0100] Retrying (Retry(total=18, connect=False, read=8, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:24:00 +0100] Retrying (Retry(total=17, connect=False, read=7, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:24:01 +0100] Retrying (Retry(total=17, connect=False, read=7, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:24:01 +0100] Retrying (Retry(total=17, connect=False, read=7, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:29:06 +0100] Retrying (Retry(total=16, connect=False, read=6, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:29:09 +0100] Retrying (Retry(total=16, connect=False, read=6, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:29:11 +0100] Retrying (Retry(total=16, connect=False, read=6, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:34:17 +0100] Retrying (Retry(total=15, connect=False, read=5, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:34:18 +0100] Retrying (Retry(total=15, connect=False, read=5, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:34:20 +0100] Retrying (Retry(total=15, connect=False, read=5, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:39:37 +0100] Retrying (Retry(total=14, connect=False, read=4, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:39:37 +0100] Retrying (Retry(total=14, connect=False, read=4, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:39:40 +0100] Retrying (Retry(total=14, connect=False, read=4, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:45:15 +0100] Retrying (Retry(total=13, connect=False, read=3, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:45:17 +0100] Retrying (Retry(total=13, connect=False, read=3, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:45:20 +0100] Retrying (Retry(total=13, connect=False, read=3, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:51:33 +0100] Retrying (Retry(total=12, connect=False, read=2, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:51:34 +0100] Retrying (Retry(total=12, connect=False, read=2, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:51:41 +0100] Retrying (Retry(total=12, connect=False, read=2, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:58:35 +0100] Retrying (Retry(total=11, connect=False, read=1, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:58:36 +0100] Retrying (Retry(total=11, connect=False, read=1, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 01:58:43 +0100] Retrying (Retry(total=11, connect=False, read=1, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 02:05:36 +0100] Retrying (Retry(total=10, connect=False, read=0, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 02:05:37 +0100] Retrying (Retry(total=10, connect=False, read=0, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain [2023-07-22 02:05:44 +0100] Retrying (Retry(total=10, connect=False, read=0, redirect=None, status=10)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /v2/files/EGAF00006161731?destinationFormat=plain 0%| | 0.00/2.89G [1:01:51<?, ?B/s] [2023-07-22 02:10:45 +0100] HTTPSConnectionPool(host='ega.ebi.ac.uk', port=8443): Max retries exceeded with url: /v2/files/EGAF00006161731?destinationFormat=plain (Caused by ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))) Traceback (most recent call last): File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "C:\Users\leej1\Anaconda3\lib\http\client.py", line 1377, in getresponse response.begin() File "C:\Users\leej1\Anaconda3\lib\http\client.py", line 320, in begin version, status, reason = self._read_status() File "C:\Users\leej1\Anaconda3\lib\http\client.py", line 281, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "C:\Users\leej1\Anaconda3\lib\socket.py", line 704, in readinto return self._sock.recv_into(b) File "C:\Users\leej1\Anaconda3\lib\ssl.py", line 1242, in recv_into return self.read(nbytes, buffer) File "C:\Users\leej1\Anaconda3\lib\ssl.py", line 1100, in read return self._sslobj.read(len, buffer) ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\adapters.py", line 439, in send resp = conn.urlopen( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen return self.urlopen( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen return self.urlopen( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 783, in urlopen return self.urlopen( [Previous line repeated 7 more times] File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen retries = retries.increment( File "C:\Users\leej1\Anaconda3\lib\site-packages\urllib3\util\retry.py", line 574, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='ega.ebi.ac.uk', port=8443): Max retries exceeded with url: /v2/files/EGAF00006161731?destinationFormat=plain (Caused by ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_file.py", line 323, in download_file_retry self.download_file(output_file, num_connections, max_slice_size) File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_file.py", line 159, in download_file for part_file_name in executor.map(self.download_fileslice, params): File "C:\Users\leej1\Anaconda3\lib\concurrent\futures_base.py", line 609, in result_iterator yield fs.pop().result() File "C:\Users\leej1\Anaconda3\lib\concurrent\futures_base.py", line 446, in result return self.get_result() File "C:\Users\leej1\Anaconda3\lib\concurrent\futures_base.py", line 391, in get_result raise self._exception File "C:\Users\leej1\Anaconda3\lib\concurrent\futures\thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_file.py", line 189, in download_fileslice return self.download_file_slice(args) File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_file.py", line 221, in download_file_slice with self.data_client.get_stream(path, extra_headers) as r: File "C:\Users\leej1\Anaconda3\lib\contextlib.py", line 119, in enter return next(self.gen) File "C:\Users\leej1\Anaconda3\lib\site-packages\pyega3\libs\data_client.py", line 76, in get_stream with self.session.get(url, headers=headers, stream=True, timeout=request_timeout_in_sec) as r: File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\sessions.py", line 555, in get return self.request('GET', url, kwargs) File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\sessions.py", line 542, in request resp = self.send(prep, send_kwargs) File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "C:\Users\leej1\Anaconda3\lib\site-packages\requests\adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='ega.ebi.ac.uk', port=8443): Max retries exceeded with url: /v2/files/EGAF00006161731?destinationFormat=plain (Caused by ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))) [2023-07-22 02:11:45 +0100] retry attempt 2 [2023-07-22 02:11:45 +0100] Download starting [using 40 connection(s), file size 2894525773 and chunk length 1073741824]... 0%| | 0.00/2.89G [00:00<?, ?B/s][2023-07-22 02:11:46 +0100] [2023-07-22 02:11:46 +0100] [2023-07-22 02:11:46 +0100] Authentication success for user 'james.lee@crick.ac.uk' [2023-07-22 02:11:46 +0100] [2023-07-22 02:11:46 +0100] Authentication success for user 'james.lee@crick.ac.uk' [2023-07-22 02:11:46 +0100] Authentication success for user 'james.lee@crick.ac.uk'

[sunnaa0423]

I can confirm having the same problem.

[LuoPangpang]

Same problem confirmed.

[CsabaHalmagyi]

Dear @JamesLeeLab Thank you for submitting the error log. I can confirm that this is an issue we are investigating currently with the development team.

[niuruize]

I can confirm having the same problem. What should I do now?

[kevingeo11]

Same problem confirmed

[HongkePn]

Just wondering if you solve the problem. I am stuck here for a month now

[niuruize]

Just wondering if you solve the problem. I am stuck here for a month now

Software bugs. I've downloaded 99% of the data, but it still fails. It's still unresolved.

[CsabaHalmagyi]

@JamesLeeLab Could you please confirm if this issue still exist? We redeployed the service last week and downloading should be working now.

[CsabaHalmagyi]

Also, may I suggest to use less connections (the log shows you were using 40), please try downloading with max 5.

[JamesLeeLab]

Now working - thanks.

BW James

From: Csaba Halmagyi @.> Sent: 25 September 2023 13:52 To: EGA-archive/ega-download-client @.> Cc: James Lee @.>; Mention @.> Subject: Re: [EGA-archive/ega-download-client] Repeated aborted connections - forcibly closed. (Issue #199)

External Sender: Use caution.

@JamesLeeLabhttps://github.com/JamesLeeLab Could you please confirm if this issue still exist? We redeployed the service last week and downloading should be working now.

- Reply to this email directly, view it on GitHubhttps://github.com/EGA-archive/ega-download-client/issues/199#issuecomment-1733651518, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A3LMLOZIFZKPGFU3RXJZVT3X4F477ANCNFSM6AAAAAA2TXTXYY. You are receiving this because you were mentioned.Message ID: @.**@.>>

The Francis Crick Institute Limited is a registered charity in England and Wales no. 1140062 and a company registered in England and Wales no. 06885462, with its registered office at 1 Midland Road London NW1 1AT

[CsabaHalmagyi]

Thank you for confirming, closing the issue.