Calls to `Mint.HTTP.recv/3` hang and error when more than number of bytes remaining requested

[tomtaylor]

I'm using Mint.HTTP.recv with a passive connection to control the how quickly we consume the request as part of a GenStage producer. I'm requesting a specific number of bytes on each loop round, according to the demand on the producer.

{:ok, conn, responses} = Mint.HTTP.recv(state.conn, recv_bytes, 30_000)

This works fine, until the last loop. If I request more than the number of remaining bytes in the request, the call to recv/3 hangs, and after 5s returns:

{:error, %Mint.HTTP1{host: "example.com", port: 443, request: nil, streaming_request: nil, socket: {:sslsocket, {:gen_tcp, #Port<0.43>, :tls_connection, :undefined}, [#PID<0.727.0>, #PID<0.726.0>]}, transport: Mint.Core.Transport.SSL, mode: :passive, scheme_as_string: "https", requests: {[], []}, state: :closed, buffer: "", proxy_headers: [], private: %{}, log: true}, %Mint.TransportError{reason: :closed}, []}

If I check for the remaining number of bytes from the Content-Length header and clamp to only fetch those, it works fine, finishing on a {:done, _} response as I'd expect:

recv_bytes =
      case state.conn.request.body do
        {:content_length, bytes_left} -> min(@recv_bytes, bytes_left)
        _ -> @recv_bytes
      end

I would have assumed, although it's not documented IIRC, that calls to recv/3 for more than the number of bytes would consume the entire socket and finish as normal. Is this a bug?

[tomtaylor]

Actually I can see from the Erlang docs for both ssl and gen_tcp, that:

If Length > 0, exactly Length bytes are returned, or an error; possibly discarding less than Length bytes of data when the socket is closed from the other side.

https://www.erlang.org/doc/man/gen_tcp#recv-3

So I guess this is expected behaviour? Although what is the right way to handle an HTTP response without a Content-Length header? Or one that is shorter than the initial byte_count?

[tomtaylor]

Here's a minimal test case if it's useful: https://gist.github.com/tomtaylor/30780323285967a567eba6b8c2e23576

[whatyouhide]

@tomtaylor thanks for the report! No, this is not a bug. It's the expected behavior, and the same as :ssl.recv/3 and :gen_tcp.recv/3. I updated the docs with more clarification 🙃

[tomtaylor]

Thanks for clarifying!