Open nlykkei opened 4 years ago
This is behaving exactly as it unfortunately must until a breaking API change can be introduced. The behaviour was likely (I don't remember this specifically) introduced to raise the same exception as it used to raise before urllib3 introduced finer-grained exceptions. Is that great? No. Is it what will happen until a real requests 3.0 can happen? Almost certainly due to backwards compatibility concerns and API stability.
@simonvanderveldt So you're complaining about exception wrapping which is a user feature to allow them to not have to think about handling exceptions from N libraries being used inside of Requests? That's tangential to this issue. Please let's not muddy the water with this conversation
I am following this issue as I also get mixed results when a timeout is raised.
My timeout is set to (1,10)
connect/read and when it times out, I can see any of these 2:
HTTPSConnectionPool(host='cloud-collector.newrelic.com', port=443): Read timed out. (read timeout=10)
HTTPSConnectionPool(host='cloud-collector.newrelic.com', port=443): Read timed out. (read timeout=1)
I imagine the second one should mention connect
instead of read
, possibly making one believe that the timeouts are not being applied correctly.
I am still trying to narrow down the problem before creating a new github issue, but it seems something isn't too clear in the way exceptions are handled.
Edit: I'm not entirely sure this is the same issue so I created https://github.com/psf/requests/issues/5544
try to enable the OS TCP/IP stack tcp keep alive, it is disable by default , ie. for Linux I use this inorder to keep the connection up...
import socket
from urllib3.connection import HTTPConnection
HTTPConnection.default_socket_options = HTTPConnection.default_socket_options + [
(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
,(socket.SOL_TCP, socket.TCP_KEEPIDLE, 45)
,(socket.SOL_TCP, socket.TCP_KEEPINTVL, 10)
,(socket.SOL_TCP, socket.TCP_KEEPCNT, 6)
]
from requests import session
self.session = session()
If the socket gets dropped because the server didnt get any tcp keep alive msg while processing your request, you will get a ConectionTimeout or ConnectionReset eror, because, it is from a lower layer than the data tranfer...so if the socket is close...no data will be tranfer...
Hello @nlykkei By your doubt I assume you're trying to handle that exception raised by urllib3, if it is, then this is what I have done to handle it.
Instead of using except ReadTimeout as e:
I used except requests.ReadTimeout as e:
and it handled just perfectly.
Please let me know if you had the same problem or not and whether you have managed to solve your problem.
Hello @nlykkei By your doubt I assume you're trying to handle that exception raised by urllib3, if it is, then this is what I have done to handle it. Instead of using
except ReadTimeout as e:
I usedexcept requests.ReadTimeout as e:
and it handled just perfectly. Please let me know if you had the same problem or not and whether you have managed to solve your problem.
Well,your are correct about the Exception handle...but, the real issue, is that the OS libraries, no matter is your request forces to use persistenr connections by enabling theConnection : Keep-Alive header on HTTP/HTTPS Request....it depends on 2 possible scenarios, 1st, the easy one... the server responds back with a connection close or a the enconding is set too chunk-response.... to resolve this just iterate the response till you get Len 0 onthe responseheader...
But At least on my findings, the major issue deepnds on the lower layers, since the content providers dont want to spend resources on "probably" dead connection, they close them before the IETFs RFC standrard,that defines the TCP keep alive behaivor.....and set it to start sendind L4 TCP KA, after 2 hours....so it is insane, and obvously will spend resourses...so what I have done is set the SO_KEEPALIVE flag on, and ser the timers lower....so youcan release the connection to the connection pool, and release resourses on your device, when idle.
I dont like catch exceptions and do retries, when they are not neccesary....
but you can retry on your code, dont set the retry param, because you willgetthe same behaivor, sincethe Lower ayer will retry the High Level Request on the same dead connection....
Cheers from Mexico
I faced the same issue, by removing the headers worked for me.
Same issue in using a custom adapter as well where the underlying urllib3 adapter can return a urllib3.exceptions.ReadTimeoutError
as a reason for MaxRetryError
but that check is missing so it falls through to a ConnectionError
https://github.com/psf/requests/blob/2d2447e210cf0b9e8c7484bfc6f158de9b24c171/requests/adapters.py#L501-L517
Please have a look at https://github.com/psf/requests/issues/4590 as the issue may be in urllib3. If not exactly this issue a very similar one. Thanks.
Please I need help. I've been trying to download spacy English package using: python -m spacy download en_core_web_sm
But I keep getting this error: requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Read timed out. (read timeout=None)
The full error message is presented below: C:\Users\user>python -m spacy download en_core_web_sm Traceback (most recent call last): File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 467, in _make_request self._validate_conn(conn) File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 1096, in _validate_conn conn.connect() File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 642, in connect sock_and_verified = _ssl_wrap_socket_and_match_hostname( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 782, in _ssl_wrap_socket_and_match_hostname ssl_sock = ssl_wrapsocket( ^^^^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\ssl.py", line 470, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, serverhostname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\ssl.py", line 514, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 517, in wrap_socket return self.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 1075, in _create self.do_handshake() File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 1346, in do_handshake self._sslobj.do_handshake() TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 486, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 844, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\retry.py", line 470, in increment raise reraise(type(error), error, _stacktrace) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\util.py", line 39, in reraise raise value File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 790, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 491, in _make_request raise new_e File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 469, in _make_request self._raise_timeout(err=e, url=url, timeout_value=conn.timeout) File "C:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 370, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Read timed out. (read timeout=None)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "
Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 536, in _make_request response = conn.getresponse() File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 464, in getresponse httplib_response = super().getresponse() File "/usr/local/lib/python3.10/http/client.py", line 1375, in getresponse response.begin() File "/usr/local/lib/python3.10/http/client.py", line 318, in begin version, status, reason = self._read_status() File "/usr/local/lib/python3.10/http/client.py", line 279, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/local/lib/python3.10/socket.py", line 705, in readinto return self._sock.recv_into(b) File "/usr/local/lib/python3.10/ssl.py", line 1307, in recv_into return self.read(nbytes, buffer) File "/usr/local/lib/python3.10/ssl.py", line 1163, in read return self._sslobj.read(len, buffer) TimeoutError: The read operation timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 843, in urlopen retries = retries.increment( File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 474, in increment raise reraise(type(error), error, _stacktrace) File "/usr/local/lib/python3.10/site-packages/urllib3/util/util.py", line 39, in reraise raise value File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 789, in urlopen response = self._make_request( File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 538, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 369, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Read timed out. (read timeout=60)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/app/app.py", line 1011, in gcs_upload response = gcp_client.upload(local_path,destination_path) File "/app/app.py", line 270, in upload blob.upload_from_filename(source_file_name) File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/blob.py", line 2947, in upload_from_filename self._handle_filename_and_upload( File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/blob.py", line 2826, in _handle_filename_and_upload self._prep_and_do_upload( File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/blob.py", line 2643, in _prep_and_do_upload created_json = self._do_upload( File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/blob.py", line 2466, in _do_upload response = self._do_resumable_upload( File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/blob.py", line 2302, in _do_resumable_upload response = upload.transmit_next_chunk(transport, timeout=timeout) File "/usr/local/lib/python3.10/site-packages/google/resumable_media/requests/upload.py", line 515, in transmit_next_chunk return _request_helpers.wait_and_retry( File "/usr/local/lib/python3.10/site-packages/google/resumable_media/requests/_request_helpers.py", line 178, in wait_and_retry raise error File "/usr/local/lib/python3.10/site-packages/google/resumable_media/requests/_request_helpers.py", line 155, in wait_and_retry response = func() File "/usr/local/lib/python3.10/site-packages/google/resumable_media/requests/upload.py", line 507, in retriable_request result = transport.request( File "/usr/local/lib/python3.10/site-packages/google/auth/transport/requests.py", line 538, in request response = super(AuthorizedSession, self).request( File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 713, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Read timed out. (read timeout=60)
How to solve this error
Consider the code below (main.py). When a temporary network disconnect occurs without the
timeout
keyword argument toSession.get()
, the client may hang indefinitely and no exception is raised.However, if I use the
timeout
keyword argument, the application will raise aConnectionError
from models.py corresponding to theurllib3.exceptions.ReadTimeoutError
:requests.exceptions.ConnectionError: HTTPSConnectionPool(host='confluence.example.net', port=443): Read timed out.
Given that the exception is only raised, when using the
timeout
keyword argument, why isn't Requests raisingReadTimeout
exception instead? In particular, theConnectionError
's exception message "Read timed out." suggests that it should be aReadTimeout
exception?To mitigate the issue I'm currently performing a regular expression match on the exception message, which is a bad practice:
main.py:
models.py: https://github.com/psf/requests/blob/master/requests/models.py
Exception:
Expected Result
I would expect a
requests.exceptions.ReadTimeout
to be raised.Actual Result
A
requests.exceptions.ConnectError
was raised instead, with the error message:Read timed out.
System Information
This command is only available on Requests v2.16.4 and greater. Otherwise, please provide some basic information about your system (Python version, operating system, &c).