httplib2.Http doesn't close socket

What steps will reproduce the problem?
1. Write a python program to create a lot of httplib2.Http objects, each makes 
a request.

What is the expected output? What do you see instead?

Each httplib2.Http create a socket and the socket will not be closed. That 
leads to the process eventually crash due to too many open files.

What version of the product are you using? On what operating system?

Latest version httplib2 and Linux OS.

Please provide any additional information below.

My python program use httplib2.Http to make http request. Once I need to 
generate a request, I create a httplib2.Http object, so that my program will 
frequently create/destroy httplib2.Http objects.

I found my program was easy to crash due to reach max number of open files. 
Checking /proc/<pid>/fd, there were too many open socket fds. The problem made 
me to have to dig into httplib2 source code.

Then I found that, in httplib2.Http._conn_request method, there was code like 
this:

        else:
            content = ""
            if method == "HEAD":
                conn.close()
            else:
                content = response.read()
            response = Response(response)
            if method != "HEAD":
                content = _decompressContent(response, content)
        break

This shows that socket is only closed when http method is HEAD. Maybe httplib2 
wanted somehow to reuse the socket. But Http class doesn't have a close() 
method. That mean when I makes a Http request, the socket will not close until 
my process terminates.

Then I modified the code:

        else:
            content = ""
            if method == "HEAD":
                conn.close()
            else:
                content = response.read()
            response = Response(response)
            if method != "HEAD":
                content = _decompressContent(response, content)
                conn.close() # I ADD THIS CLOSE
        break

After that, my program worked well.

But I'm still curious if that's really httplib2's bug given that httplib2 is a 
very old and common lib.
Original issue reported on code.google.com by li.evan....@gmail.com on 22 May 2013 at 8:37
jirumiro / httplib2

httplib2.Http doesn't close socket #285
