python-hyper / hyper

HTTP/2 for Python.
http://hyper.rtfd.org/en/latest/
MIT License
1.05k stars 191 forks source link

Can I set the value time out #318

Closed mingjunyang closed 7 years ago

mingjunyang commented 7 years ago

I have a long time request , more than 20 second. but it's raise time out error.

I study this document,but I don't know how to set time out .

who can help me?

# MacOS laster version
# python3 --version
# Python 3.6.1
# pyOpenSSL (16.2.0)
# hyper (0.7.0)
from hyper import HTTP20Connection, HTTPConnection, compat, tls

host = "devhost"
ssl_context = tls.init_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE

conn = HTTPConnection(host=host, port=443, ssl_context=ssl_context, secure=True )

conn.request(  'GET', "/mypath )
resp = conn.get_response()
Traceback (most recent call last):
  File "timeouttest.py", line 38, in <module>
    resp1 = conn.get_response()
  File "/usr/local/lib/python3.6/site-packages/hyper/common/connection.py", line 129, in get_response
    return self._conn.get_response(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/hyper/http20/connection.py", line 312, in get_response
    return HTTP20Response(stream.getheaders(), stream)
  File "/usr/local/lib/python3.6/site-packages/hyper/http20/stream.py", line 230, in getheaders
    self._recv_cb(stream_id=self.stream_id)
  File "/usr/local/lib/python3.6/site-packages/hyper/http20/connection.py", line 771, in _recv_cb
    self._single_read()
  File "/usr/local/lib/python3.6/site-packages/hyper/http20/connection.py", line 665, in _single_read
    self._sock.fill()
  File "/usr/local/lib/python3.6/site-packages/hyper/common/bufsocket.py", line 167, in fill
    count = self._sck.recv_into(self._buffer_view[self._buffer_end:])
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1002, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 865, in read
    return self._sslobj.read(len, buffer)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 625, in read
    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
Lukasa commented 7 years ago

hyper doesn't have any timeout controls: it just uses the system default timeout. Try using socket.setdefaulttimeout.

mingjunyang commented 7 years ago

Do you mean to do this?

from hyper import HTTP20Connection, HTTPConnection, compat, tls
import socket

host = "devhost"
port = 443
ssl_context = tls.init_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE

conn = HTTPConnection(host=host, port=port, ssl_context=ssl_context, secure=True )

###### do hack
socket.setdefaulttimeout(30)
sock = socket.create_connection((host, port))
conn._sock = sock
######

conn.request(  'GET', "/mypath )
resp = conn.get_response()
Lukasa commented 7 years ago

There's no need for the patching. A call to socket.setdefaulttimeout() immediately after the imports, but before you create a HTTPConnection, will be totally sufficient.

mingjunyang commented 7 years ago

I set the socket.setdefaulttimeout(30) before import HTTPConnection. And print the variable t , the t value is 30.

###
import socket
socket.setdefaulttimeout(30)
t = socket.getdefaulttimeout()
print(t)
###

from hyper import HTTPConnection,tls

host = "devhost"
port = 443
ssl_context = tls.init_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE

start = time.time()
try:
    conn.request( 'GET', "/mypath" )
    resp = conn.get_response()
    temp_data = resp.read().decode('utf-8')
except Exception as e:
    raise
finally:
    print('time\t', time.time() - start)

but agein raise exception socket.timeout: The read operation timed out again. I print the request time, is the default 5 seconds.

30.0
time     5.025950193405151
Traceback (most recent call last):
  File "timeouttest.py", line 48, in <module>
    resp1 = conn.get_response()
  File "/usr/local/lib/python3.6/site-packages/hyper/common/connection.py", line 129, in get_response
    return self._conn.get_response(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/hyper/http20/connection.py", line 312, in get_response
    return HTTP20Response(stream.getheaders(), stream)
  File "/usr/local/lib/python3.6/site-packages/hyper/http20/stream.py", line 230, in getheaders
    self._recv_cb(stream_id=self.stream_id)
  File "/usr/local/lib/python3.6/site-packages/hyper/http20/connection.py", line 771, in _recv_cb
    self._single_read()
  File "/usr/local/lib/python3.6/site-packages/hyper/http20/connection.py", line 665, in _single_read
    self._sock.fill()
  File "/usr/local/lib/python3.6/site-packages/hyper/common/bufsocket.py", line 167, in fill
    count = self._sck.recv_into(self._buffer_view[self._buffer_end:])
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1002, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 865, in read
    return self._sslobj.read(len, buffer)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 625, in read
    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

I study the source connection.py , I hack edit the source line 123 sock = socket.create_connection((host, port), 5) to sock = socket.create_connection((host, port)) on my local, the request time out more than 5 seconds and not raise timeout now, my issue solved.

maybe this is the issue.

  def connect(self):
        """
        Connect to the server specified when the object was created. This is a
        no-op if we're already connected.
        :returns: Nothing.
        """
        if self._sock is None:
            if not self.proxy_host:
                host = self.host
                port = self.port
            else:
                host = self.proxy_host
                port = self.proxy_port
            ### !!!!! I hack here
            # sock = socket.create_connection((host, port), 5)
            sock = socket.create_connection((host, port) )
            ###
            proto = None

            if self.secure:
                assert not self.proxy_host, "Proxy with HTTPS not supported."
                sock, proto = wrap_socket(sock, host, self.ssl_context)

            log.debug("Selected protocol: %s", proto)
            sock = BufferedSocket(sock, self.network_buffer_size)

            if proto not in ('http/1.1', None):
                raise TLSUpgrade(proto, sock)

            self._sock = sock

        return
Lukasa commented 7 years ago

Ah I see, you're timing out in the connection handshake. Yes, in that case you do need to make the monkeypatch in your original response to adjust the timeout on connection setup.