Connection timeout during request phase raises "AttributeError: 'NoneType' object has no attribute 'makefile'"

What steps will reproduce the problem?

Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import httplib2
>>> h = httplib2.Http(timeout=0.1)
>>> h.request('http://www.twitlonger.com/show/8prp95')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/markpasc/Work/httplib2-timeout/httplib2/python2/httplib2/__init__.py", line 1156, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/Users/markpasc/Work/httplib2-timeout/httplib2/python2/httplib2/__init__.py", line 928, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/Users/markpasc/Work/httplib2-timeout/httplib2/python2/httplib2/__init__.py", line 898, in _conn_request
    response = conn.getresponse()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 948, in getresponse
    method=self._method)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 329, in __init__
    self.fp = sock.makefile('rb', 0)
AttributeError: 'NoneType' object has no attribute 'makefile'
>>> 

What is the expected output? What do you see instead?

I expected to receive the underlying socket.timeout exception, not the observed 
AttributeError.

If I apply the attached patch, it appears like the AttributeError is a masking 
error as in issue #62. The stack trace in that case is:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/markpasc/Work/httplib2-timeout/httplib2/python2/httplib2/__init__.py", line 1158, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/Users/markpasc/Work/httplib2-timeout/httplib2/python2/httplib2/__init__.py", line 930, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/Users/markpasc/Work/httplib2-timeout/httplib2/python2/httplib2/__init__.py", line 869, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 874, in request
    self._send_request(method, url, body, headers)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 911, in _send_request
    self.endheaders()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 868, in endheaders
    self._send_output()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 740, in _send_output
    self.send(msg)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 699, in send
    self.connect()
  File "/Users/markpasc/Work/httplib2-timeout/httplib2/python2/httplib2/__init__.py", line 756, in connect
    raise socket.error, msg
socket.timeout: timed out

The underlying socket.timeout exception is raised by the conn.request() at line 
869 but then, as in issue #62, caught in the "except socket.error, e:" at line 
873 (as socket.timeout is-a socket.error) so it can attempt to 
conn.getresponse() anyway, which raises the masking AttributeError.

This timeout error only reproduces against a slow website, as the timeout has 
to occur while it's still connecting or sending the request. The given 
twitlonger.com URL is the real one I saw in production, and currently 
reproduces the error for me (though not at the production timeout of 10). If I 
try with a faster site like google.com, the program is already down in 
conn.getresponse() by the time the timeout occurs, and the exception is handled 
differently by the different code path.

If I use timeout=0, I still get an AttributeError, but the masked error I 
observe is a socket.error: "Operation now in progress" (errno=EINPROGRESS) 
instead.

(I'm not at all sure how to include tests for these, since you're testing 
against real resources, and with timeout.cgi the CGI machinery is accepting the 
request so the test has no control.)

What version of the product are you using? On what operating system?

This occurs for me using httplib2 revision 4605b4c250b5, including yesterday's 
changesets to fix issue #96. (As shown above, I'm using Python 2.6.1 for Mac OS 
X 10.6.)

Please provide any additional information below.

The similar issue #105 seems to be another dup of issue #96 (since issue #102 
was marked that way).
Original issue reported on code.google.com by markpaaa...@gmail.com on 13 Feb 2011 at 8:54
Attachments:
catch-socket-timeout-patch.txt
eshad / httplib2

Connection timeout during request phase raises "AttributeError: 'NoneType' object has no attribute 'makefile'" #131
