benoitc / hackney

simple HTTP client in Erlang
Other
1.34k stars 427 forks source link

Unexpected `enomem` error during file downloading #744

Open viralpraxis opened 3 months ago

viralpraxis commented 3 months ago

Hi there!

We tried to migrate from 1.18.1 to 1.20.1 and encountered increased error rate during downloading huge (~1GB) files.

I've published an MRE: https://gist.github.com/viralpraxis/6209746ee3b108473452ee810678f769 You can run it via mix run -e 'Runner.start($URI)', $URI points to a file over 1GB (serving over python -m http.server would be fine).

On 1.18.1:

21:47:47.878 [info] 2: Streaming completed
21:47:47.946 [info] 1: Streaming completed
21:47:47.990 [info] 5: Streaming completed
21:47:48.055 [info] 4: Streaming completed
21:47:48.066 [info] 3: Streaming completed

On 1.20.1:

21:48:12.494 [error] Error while streaming: enomem
21:48:12.494 [error] Error while streaming: enomem
21:48:12.494 [error] Error while streaming: enomem
21:48:12.494 [error] Error while streaming: enomem
21:48:12.494 [error] Error while streaming: enomem

In fact this error is reproducible on 1.18.2. I bisect'd this error to https://github.com/benoitc/hackney/commit/5e74354a48653fe2456688f80c6bccb11143f6af (previous commit https://github.com/benoitc/hackney/commit/e3872f768a4f0b74c20a03c5e23ea9652d811f0e works fine).

elixir 1.16.1
erlang 26.2.2

Please let my know if you need any addition information. Thanks!