Add support for libcurl's multi interface

gryumov commented 2 months ago

Add support for libcurl's multi event interface

Description

EasyCurl.jl currently utilizes libcurl’s ‘easy’ synchronous interface. This implies that applications using EasyCurl.jl must follow a sequential request logic, which may not always be the most efficient approach.

To address this limitation, libcurl’s multi interface comes into play. It is built on top of the ‘easy’ interface and should be relatively straightforward to integrate with the existing EasyCurl.jl package.

Given our goal of creating event-driven systems, we are particularly interested in adding the event interface implemented by curl_multi_socket_action function.

Basic requirements

[ ] The methods must confirm to the library's existing structure
[ ] The methods must be tested to cover all possible use cases, errors and mistakes
[ ] The methods must be documented in accordance with the guidelines

Tasks

[ ] Research the existing implementation of the multi event interface at Downloads.jl
[ ] Implement a similar event based on top of the existing library.
[ ] Write tests.
[ ] Benchmark the solution.

Resources
Downloads.jl (core package)
libcurl multi event tutorial
libcurl multi tutorial

bluesmoon commented 2 months ago

Would you consider joining the development of https://github.com/bluesmoon/CurlHTTP.jl instead. It already has support for CurlMulti.

dmitrii-doronin commented 2 months ago

Hi, @bluesmoon!

We already have a lot of packages (and many more coming) that already rely on EasyCurl.jl, so we can't switch easily. However, I think we could figure something out. Maybe you're willing to do sort of a merger and become one of the contributors? We can work out a plan and help with a rewrite to have this issue covered. What do you think?

bluesmoon commented 2 months ago

CurlHTTP has been around for a while. We open sourced it 8 months ago, but it has been in use internally at my company for about 3 years, and it's quite mature at this point. We also have a lot of dependence on it as well as other open source packages.

One easy way to get all the functionality into EasyCurl would be to make a EasyCurl into a wrapper around CurlHTTP. I can look into what that would take.

dmitrii-doronin commented 2 months ago

Funnily enough, EasyCurl has been around for about 6 years in our company, used around in different internal projects. We've decided to open source it not that long ago, too.

I think it's possible to do a soft merger or something like that if you wish. Supporting both interfaces for some time. We shouldn't be clashing around this package though. Nobody wants to rewrite their code base. :)

Do you have any other projects you want to open source? We've made our own registry Green because of the General's rigid policies. We're more lax on naming conventions but more stringent on cod and docs.

bluesmoon commented 2 months ago

Nice! Yes, there will be other code we open source over time. Will you be at JuliaCon this year? I'll be speaking there.

dmitrii-doronin commented 2 months ago

I can't say for sure right now, but I'll watch the talks for sure. What are you going to cover?

bluesmoon commented 2 months ago

The motivation behind CurlHTTP & the capabilities.

bhftbootcamp / EasyCurl.jl