ninenines / gun

HTTP/1.1, HTTP/2, Websocket client (and more) for Erlang/OTP.
ISC License
891 stars 232 forks source link

Can this client (or any?) make multiple concurrent outbound requests? #335

Open ahungry opened 2 months ago

ahungry commented 2 months ago

Hi, a bit new to Erlang - is it possible to start, say, 3 outbound HTTP requests simultaneously, and collect the results from the caller (such that if each remote endpoint took ~2 seconds to reply, the total duration on the calling side would be ~2 seconds, and not 6)?

I've come across erpc.multicall and rpc.pmap in the stdlib, but I'm not sure either of those fit this use case, so curious if this would be something supported in the http client(s) out there themselves? (though search matches for "concurren" seem sparse).

I did see Elixir has a new Task.await_many() which seems similar (basically, JS Promise.all() equivalent).

Sorry for the slightly off topic question, but TIA if anyone can answer.

essen commented 2 months ago

Yes and no.

Gun is asynchronous so you can do the requests all at once and gather the results afterwards:

StreamRef1 = gun:get(ConnPid, "/one"),
StreamRef2 = gun:get(ConnPid, "/two"),
StreamRef3 = gun:get(ConnPid, "/three"),
gather_results(StreamRef1, StreamRef2, StreamRef3)

Your gather_results function (which could be your main receive function) can receive the responses asynchronously if necessary. The responses and data is sent as messages.