HTTP stream/2 and recv/3 returns undocumented error tuple

[sezaru]

Mint's documentation tells that HTTP.stream/2 and HTTP.recv/3 functions can receive errors in this format: {:error, t(), Mint.Types.error(), [Mint.Types.response()]}

But looking at my logs it seems that I actually receive an error that doesn't follow this structure:

{
  :error, 
  #Reference<0.3293968231.3493068809.17166>,
  %Mint.HTTPError{module: Mint.HTTP2, reason: {:server_closed_request, :internal_error}}
}

Is the documentation incomplete regarding the possible return values of these functions?

More information:

I'm using Mint via Tesla library, I got a :function_clause error in this function:

    defp receive_packet(conn, ref, opts, acc \\ %{}) do
      with {:ok, conn, responses} <- receive_message(conn, opts),
           acc <- reduce_responses(responses, ref, acc) do
        {:ok, conn, acc}
      else
        {:error, error} ->
          if opts[:close_conn], do: {:ok, _conn} = close(conn)
          {:error, error}

        {:error, _conn, error, _res} ->
          if opts[:close_conn], do: {:ok, _conn} = close(conn)
          {:error, "Encounter Mint error #{inspect(error)}"}

        :unknown ->
          if opts[:close_conn], do: {:ok, _conn} = close(conn)
          {:error, :unknown}
      end
    end

Where receive_message is:

    defp receive_message(conn, %{mode: :active} = opts) do
      receive do
        message when is_tuple(message) and elem(message, 0) in @tags ->
          HTTP.stream(conn, message)
      after
        opts[:timeout] -> {:error, :timeout}
      end
    end

    defp receive_message(conn, %{mode: :passive} = opts),
      do: HTTP.recv(conn, 0, opts[:timeout])

Because Tesla tries to match the response of stream/2 and recv/3 with the documented possible returns. But as described, I received another response that didn't match any of the cases.

Here is the full error log in case it helps:

Erlang error: {{:function_clause, [{Tesla.Adapter.Mint, :"-reduce_responses/3-fun-2-", [{:error, #Reference<0.3293968231.3493068809.17166>, %Mint.HTTPError{module: Mint.HTTP2, reason: {:server_closed_request, :internal_error}}}, %{data: "[{\"id\":96114152,\"price\":\"3525.00000000\",\"qty\":\"0.01992300\",\"quoteQty\":\"70.22857500\",\"time\":1548127498702,\"isBuyerMaker\":true,\"isBestMatch\":true},{\"id\":96114153,\"price\":\"3525.00000000\",\"qty\":\"0.10000000\",\"quoteQty\":\"352.50000000\",\"time\":1548127499060,\"isBuyerMaker\":true,\"isBestMatch\":true},{\"id\":96114154,\"price\":\"3526.63000000\",\"qty\":\"0.00311700\",\"quoteQty\":\"10.99250571\",\"time\":1548127499080,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114155,\"price\":\"3526.60000000\",\"qty\":\"0.00297700\",\"quoteQty\":\"10.49868820\",\"time\":1548127499379,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114156,\"price\":\"3525.00000000\",\"qty\":\"0.02381900\",\"quoteQty\":\"83.96197500\",\"time\":1548127499424,\"isBuyerMaker\":true,\"isBestMatch\":true},{\"id\":96114157,\"price\":\"3526.59000000\",\"qty\":\"0.00297700\",\"quoteQty\":\"10.49865843\",\"time\":1548127499437,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114158,\"price\":\"3526.03000000\",\"qty\":\"0.18847000\",\"quoteQty\":\"664.55087410\",\"time\":1548127499438,\"isBuyerMaker\":true,\"isBestMatch\":true},{\"id\":96114159,\"price\":\"3526.58000000\",\"qty\":\"0.00297700\",\"quoteQty\":\"10.49862866\",\"time\":1548127499510,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114160,\"price\":\"3526.57000000\",\"qty\":\"0.00297700\",\"quoteQty\":\"10.49859889\",\"time\":1548127499565,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114161,\"price\":\"3526.56000000\",\"qty\":\"0.00297700\",\"quoteQty\":\"10.49856912\",\"time\":1548127499624,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114162,\"price\":\"3526.55000000\",\"qty\":\"0.01952900\",\"quoteQty\":\"68.86999495\",\"time\":1548127499717,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114163,\"price\":\"3525.03000000\",\"qty\":\"0.00297700\",\"quoteQty\":\"10.49401431\",\"time\":1548127501497,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114164,\"price\":\"3526.01000000\",\"qty\":\"0.12755900\",\"quoteQty\":\"449.77430959\",\"time\":1548127501581,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114165,\"price\":\"3525.03000000\",\"qty\":\"0.00297700\",\"quoteQty\":\"10.49401431\",\"time\":1548127502007,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114166,\"price\":\"3526.01000000\",\"qty\":\"0.01394700\",\"quoteQty\":\"49.17726147\",\"time\":1548127502007,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114167,\"price\":\"3526.01000000\",\"qty\":\"0.09268100\",\"quoteQty\":\"326.79413281\",\"time\":1548127502226,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114168,\"price\":\"3526.01000000\",\"qty\":\"0.12768000\",\"quoteQty\":\"450.20095680\",\"time\":1548127502348,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114169,\"price\":\"3525.03000000\",\"qty\":\"0.00297100\",\"quoteQty\":\"10.47286413\",\"time\":1548127502782,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114170,\"price\":\"3525.03000000\",\"qty\":\"0.00000600\",\"quoteQty\":\"0.02115018\",\"time\":1548127502988,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114171,\"price\":\"3525.03000000\",\"qty\":\"0.00297200\",\"quoteQty\":\"10.47638916\",\"time\":1548127502988,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114172,\"price\":\"3525.03000000\",\"qty\":\"0.00000500\",\"quoteQty\":\"0.01762515\",\"time\":1548127503529,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114173,\"price\":\"3525.03000000\",\"qty\":\"0.00297200\",\"quoteQty\":\"10.47638916\",\"time\":1548127503529,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114174,\"price\":\"3525.02000000\",\"qty\":\"0.26806300\",\"quoteQty\":\"944.92743626\",\"time\":1548127504162,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114175,\"price\":\"3525.03000000\",\"qty\":\"0.00000500\",\"quoteQty\":\"0.01762515\",\"time\":1548127506362,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114176,\"price\":\"3525.92000000\",\"qty\":\"0.05020400\",\"quoteQty\":\"177.01528768\",\"time\":1548127506445,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114177,\"price\":\"3525.91000000\",\"qty\":\"0.05020400\",\"quoteQty\":\"177.01478564\",\"time\":1548127506556,\"isBuyerMaker\":false,\"isBestMatch\":true},{\"id\":96114178,\"price\":\"3525.04000000\",\"qty\":\"0.00314200\",\"quoteQty\":\"11.07567568\",\"time\":1548127506775,\"isBuyerMaker\":true,\"isBestMatch\":true},{\"id\":96114179,\"price\":\"3525.04000000\",\"qty\":\"0.00334000\",\"quoteQty\":\"11.77363360\",\"time\":1548127510918,\"isBuyerMaker\":true,\"isBestMatch\":true},{\"id\":96114180,\"price\":\"3525.3300000" <> ..., headers: [{"content-type", "application/json;charset=UTF-8"}, {"content-length", "145319"}, {"date", "Sun, 21 Feb 2021 00:10:02 GMT"}, {"server", "nginx"}, {"vary", "Accept-Encoding"}, {"x-mbx-uuid", "c0288a59-2a50-4521-a706-aefd21c507e7"}, {"x-mbx-used-weight", "35"}, {"x-mbx-used-weight-1m", "35"}, {"strict-transport-security", "max-age=31536000; includeSubdomains"}, {"x-frame-options", "SAMEORIGIN"}, {"x-xss-protection", "1; mode=block"}, {"x-content-type-options", "nosniff"}, {"content-security-policy", "default-src 'self'"}, {"x-content-security-policy", "default-src 'self'"}, {"x-webkit-csp", "default-src 'self'"}, {"cache-control", "no-cache, no-store, must-revalidate"}, {"pragma", "no-cache"}, {"expires", "0"}, {"access-control-allow-origin", "*"}, {"access-control-allow-methods", "GET, HEAD, OPTIONS"}, {"x-cache", "Miss from cloudfront"}, {"via", "1.1 fa3f07f937529cd473488970e2062033.cloudfront.net (CloudFront)"}, {"x-amz-cf-pop", "SCL50-C1"}, {"x-amz-cf-id", "7PiSnu2Zdfgy2W26SbNg7talTc1W2lxsHD61t0WUK7chmAJZryaf3w=="}], status: 200}], [file: 'lib/tesla/adapter/mint.ex', line: 328]}, {Enum, :"-reduce/3-lists^foldl/2-0-", 3, [file: 'lib/enum.ex', line: 2181]}, {Tesla.Adapter.Mint, :receive_packet, 4, [file: 'lib/tesla/adapter/mint.ex', line: 298]}, {Tesla.Adapter.Mint, :receive_responses, 4, [file: 'lib/tesla/adapter/mint.ex', line: 258]}, {Tesla.Adapter.Mint, :format_response, 3, [file: 'lib/tesla/adapter/mint.ex', line: 197]}, {Tesla.Adapter.Mint, :call, 2, [file: 'lib/tesla/adapter/mint.ex', line: 59]}, {Tesla.Middleware.JSON, :call, 3, [file: 'lib/tesla/middleware/json.ex', line: 54]}, {RateLimit.Impl, :"-do_run_async/4-fun-0-", 4, [file: 'lib/rate_limit/impl.ex', line: 45]}]}, {GenServer, :call, [RateLimit.Server.Binance, {:run, #Function<6.129655239/1 in Historic.DownloadManager.Adapters.Binance.do_download/2>}, :infinity]}}

[whatyouhide]

@sezaru the error you get is a response, not the return value of Mint.HTTP.stream/2 or Mint.HTTP.recv/3. The function clause error you pasted at the bottom shows you that it's happening not in the code that you pasted, but rather in the reduce_responses/3 function:

{:function_clause, [{Tesla.Adapter.Mint, :"-reduce_responses/3-fun-2-", [{:error, #Reference<0.3293968231.3493068809.17166>, %Mint.HTTPError{module: Mint.HTTP2, reason: {:server_closed_request, :internal_error}}} ...

I’m gonna close this issue as I’m pretty sure it's not an issue in Mint but please feel free to comment or reopen if you can't find the fix in Tesla.Adapter.Mint.reduce_responses/3. :)

[whatyouhide]

@sezaru actually that was pretty straightforward to find, here's the problem:

https://github.com/teamon/tesla/blob/cc18e12226ef5b61f19daf7cc85cb8f94ae428b1/lib/tesla/adapter/mint.ex#L327-L340

As you can see there's a clause missing for the {:error, ref, reason} possible response (see https://hexdocs.pm/mint/1.2.1/Mint.Types.html#t:response/0).

[sezaru]

Yep, you are right, I didn't see that the error was related to the response() type instead of the tuple stream/2 and recv/3 receives.. I guess is not a good idea to open issues in Github at 2 AM :facepalm: