search

sneako / finch

Elixir HTTP client, focused on performance
MIT License
1.23k stars 114 forks source link

Do not use Finch.async_request with Stream #235

Closed hissssst closed 11 months ago

hissssst commented 11 months ago

I can see that you've added Finch.async_request and doc even states this as a way to go to use it with Stream.resource, but

The biggest problem

It doesn't lazily read the data from socket on demand, it just sends whole response body into message queue which can just overflow the server. Any huge (let's say 1TB) response will just shut down any program using this approach. And people usually use Stream for these kind of problems (requesting huge files). So this is just misleading and has nothing to do with stream, please remove this from your documentation

josevalim commented 11 months ago

We should mention that async_request works as a firehose, it will immediately send messages as they come to the current process. I think discouraging its usage with Stream is fine. And we should mention Finch.stream as an option to lazily stream.