search

toland / patron

Ruby HTTP client based on libcurl
http://toland.github.com/patron/
MIT License
541 stars 73 forks source link

Connection pool in patron #97

Closed sathishceg closed 8 years ago

sathishceg commented 8 years ago

Is it possible to have connection pooling in patron?? configuring like below:

Patron.configure do |config| config.size = 5 config.pool_timeout = 5 config.keep_alive_timeout = 30 end

toland commented 8 years ago

So, this is possible, but not easy. It has been quite a while since I have actively worked on Patron or used libcurl, but based on my reading of the libcurl docs a naive implementation of a pool would still run serially. In order to get concurrency, we would have to add support for curl_multi and I have no idea how big of a project this would be.

I think this would be a cool project, but I don't have any free time to tackle this right now.

julik commented 8 years ago

@sathishceg Have you experimented with the connection_pool gem by Mike Perham for this? Even though CURL provides curl_multi (that Patron does not support) the actual request execution method does have the right GIL guards. This means that Patron's CURL requests should multiplex nicely (even though it would not use pipelining and the like). This might provide a nice alternative until Patron supports such things natively (disclaimer: we are using Patron in a massively multithreaded context and not finding too many issues, but the threading is controlled outside of it and there is one Session per thread).

julik commented 8 years ago

A very interesting project but for now it's better to use an external connection pool, and it is now written up in the docs. Keep-alive is unfortunately not supported until we support curl_multi, which is a special endeavour all on it's own.