search

toland / patron

Ruby HTTP client based on libcurl
http://toland.github.com/patron/
MIT License
541 stars 73 forks source link

Idea: Connection pooling using curl_multi #138

Open toland opened 8 years ago

toland commented 8 years ago

This idea came up in the discussion for #97. The idea would be to use curl_multi under the hood for asynchronous HTTP operations and then add a simple pool (perhaps using a well-known gem) on top. I am not sure if this is a very good idea, or how difficult it would be; so, for now, this is in the "maybe" pile.

curl_multi docs: http://curl.haxx.se/libcurl/c/libcurl-multi.html

julik commented 8 years ago

I did think about this a little bit but I couldn't arrive at a solution that would not involve callbacks. Curious if you have an API sketch in mind. Maybe we should also see how Curb does it - they have it made ;-)

toland commented 8 years ago

Curb was my original inspiration for Patron, way back in the day. :)

I hadn't got down to details yet, but I was kicking around the idea of using futures. I don't like the idea of callbacks. I will see what curb is doing.

On September 7, 2016 at 4:29:37 AM, Julik Tarkhanov ( notifications@github.com) wrote:

I did think about this a little bit but I couldn't arrive at a solution that would not involve callbacks. Curious if you have an API sketch in mind. Maybe we should also see how Curb does it - they have it made ;-)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/toland/patron/issues/138#issuecomment-245226190, or mute the thread https://github.com/notifications/unsubscribe-auth/AAASij1PRR52yAodDgpHLb4oCMkoM_87ks5qnoQBgaJpZM4J2aPX .

julik commented 8 years ago

@toland if you feel like picking up Ruby again this would be rad to have. We are currently using Patron sessions isolated to threads, pretty much universally and in multiple systems - that works great. I can imagine that using curl_multi can bring benefits. I don't think I have time to add curl_multi support at this time - specifically because we do not need it that much. What I would very much like to try instead is using Patron as transport for the AWS SDK. This requires streaming support that Curb does have but Patron doesn't, so I think the next big thing on Patron for me will be adding block-yielding streaming.

toland commented 8 years ago

@julik My life has been quite busy lately. I would like to do this, but I'm not sure when I will be able to get around to it. I don't think it is necessarily a "must have," for the reasons you pointed out, but it would be very nice. I think the streaming support is a great addition, and I encourage you to tackle that if you have time.

I will still be available to review code, cut releases, etc.