Open ccoenen opened 8 years ago
These are the main triggers for closing a persistent connection:
https://github.com/httprb/http/blob/master/lib/http/client.rb#L95
Does this happen every time, or is it spurious? If it's spurious, it's probably http.rb trying to handle and gracefully recover from closed connection errors
As far as i can tell from wireshark, I guess this connection close happens every time. I'll look into the triggers a little later, I am on a different project this week.
The codepaths I linked are the only ones I'm aware of that would trigger a connection close. Perhaps you're trying to reuse a connection that's in a dirty state? (although looking at your code, I'm not sure where that would happen)
Ran into the same issue today. Here's what I believe is happening:
HTTP::Chainable#headers
is invoked.HTTP::Chainable#headers
invokes HTTP::Chainable#branch
, which creates a fresh HTTP::Client
and a fresh socket.Essentially a new "persistent" connection is created (and closed) for each request when using one of the HTTP::Chainable
methods (save for #request
which is implemented directly on HTTP::Client
). To get the behaviour you want, you'll have to pass the headers to #get
directly: response = @http.get(url, params: query, headers: accept: "application/json")
.
It would be good to document this very clearly in the wiki page on persistent connections, since it's not at all obvious this is what's going on.
@britishtea Thank you for your comment describing the issue. I just started using this lib and was getting quite frustrated trying to figure out why persistent connections where not working.
This opens a multiple connection (in my app so many it exhausted the max file descriptors!)
conn = HTTP.persistent('http://example.com')
(1..20).map do |i|
conn.basic_auth(:user => username, :pass => password)
.post('/post_path/', "Number #{i}")
end
whilst this seem to work as expected:
conn = HTTP.persistent('http://example.com')
.basic_auth(:user => username, :pass => password)
(1..20).map { |i| conn.post('/post_path/', "Number #{i}") }
I have recorded this in wireshark. I am talking to an API (Rails, delivered via NGINX). For some reason, the persistent connection is not persistent:
It is a chunked response, that I can parse correctly. But Right after the response, my http.rb client will send a
FIN
TCP-Packet, closing the connection. This happens in Frame 47386 of this following screenshot.I am not (to my knowledge) closing the connection manually. I am using the HTTP.persistent API. I am writing a line to a logfile when I create such a persistent object, There's the expected amount of lines in that logfile. However, Wireshark reports lots and lots of un-reused http connections.
Here's the bit of code, which is involved. It's wrapped into a https://github.com/mperham/connection_pool Which is being used from a Celluloid application.
I would appreciate pointers where to start debugging.