get_file_content fails to download some packages

[idan]

I discovered this issue when trying to pip install ipython on a virgin homebrew python2 (2.7.5 on OSX), after upgrading pip to 1.4.1.

pip install ipython consistently downloaded about 9.0 of 9.2mb before timing out with

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/pip/download.py", line 89, in get_file_content
    return geturl(resp), resp.read().decode(encoding)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 351, in read
    data = self._sock.recv(rbufsize)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 567, in read
    s = self.fp.read(amt)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 380, in read
    data = self._sock.recv(left)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 241, in recv
    return self.read(buflen)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 160, in read
    return self._sslobj.read(len)
socket.error: [Errno 54] Connection reset by peer
>>>

Both curl and wget fetch the file just fine. The specific URL that pip tried to fetch was https://pypi.python.org/packages/source/i/ipython/ipython-1.0.0.zip#md5=931bb0cd54d83316f8e9640292496c22.

I later pip installed requests (was present in pip cache), and tried to reqeusts.get() the same URL. It didn't have any trouble GET'ing the file.

Seems like a pip bug, or something in the underlying libraries. Chatted with @dstufft about it, didn't find any simple answers.

[pnasrat]

Can you try this smaller testcase either as a script or in an interpreter and see if you get the socket error?

from pip.download import urlopen
r = urlopen('https://pypi.python.org/packages/source/i/ipython/ipython-1.0.0.zip#md5=931bb0cd54d83316f8e9640292496c22')
r.read()

[pnasrat]

FWIW I just ran here on OS X 10.8.4 with homebrew python and pip 1.4.1 (although in a virtualenv)

pip install ipython
Downloading/unpacking ipython
  Downloading ipython-1.0.0.zip (9.2MB): 9.2MB downloaded
  Running setup.py egg_info for package ipython
    running egg_info

Installing collected packages: ipython
  Running setup.py install for ipython
    running install

    Installing ipcontroller script to /Users/pnasrat/benv/bin
    Installing iptest script to /Users/pnasrat/benv/bin
    Installing ipcluster script to /Users/pnasrat/benv/bin
    Installing ipython script to /Users/pnasrat/benv/bin
    Installing pycolor script to /Users/pnasrat/benv/bin
    Installing iplogger script to /Users/pnasrat/benv/bin
    Installing irunner script to /Users/pnasrat/benv/bin
    Installing ipengine script to /Users/pnasrat/benv/bin
Successfully installed ipython
Cleaning up...

[dstufft]

I believe this should be fixed in #1145

[dstufft]

I'm going to guess this problem has been resolved with the switch to requests. If this is still affecting you please comment back and we'll reopen the issue.

[dov]

I was just hit by the same issue under Fedora 20:

IPython 2.0.0-dev -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from pip.download import urlopen

In [2]: r = urlopen('https://pypi.python.org/packages/source/i/ipython/ipython-1.0.0.zip#md5=931bb0cd54d83316f8e9640292496c22')

In [3]: r.read()
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-3-e76d0b86a0e5> in <module>()
----> 1 r.read()

/usr/lib/python2.7/socket.pyc in read(self, size)
    349             while True:
    350                 try:
--> 351                     data = self._sock.recv(rbufsize)
    352                 except error, e:
    353                     if e.args[0] == EINTR:

/usr/lib/python2.7/httplib.pyc in read(self, amt)
    565         # connection, and the user is reading more bytes than will be provided
    566         # (for example, reading in 1k chunks)
--> 567         s = self.fp.read(amt)
    568         if not s:
    569             # Ideally, we would raise IncompleteRead if the content-length

/usr/lib/python2.7/socket.pyc in read(self, size)
    378                 # fragmentation issues on many platforms.
    379                 try:
--> 380                     data = self._sock.recv(left)
    381                 except error, e:
    382                     if e.args[0] == EINTR:

/usr/lib/python2.7/ssl.pyc in recv(self, buflen, flags)
    239                     "non-zero flags not allowed in calls to recv() on %s" %
    240                     self.__class__)
--> 241             return self.read(buflen)
    242         else:
    243             return self._sock.recv(buflen, flags)

/usr/lib/python2.7/ssl.pyc in read(self, len)
    158 
    159         try:
--> 160             return self._sslobj.read(len)
    161         except SSLError, x:
    162             if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:

error: [Errno 104] Connection reset by peer

In [4]: print pip.__version__
1.4.1