Unexpected unwrapped return `%Mint.TransportError{reason: :closed}` from `Mint.HTTP1.connect/4`

[jozuas]

• Req version: 0.5.1
• Finch version: 0.18.0
• Mint version: 1.6.1
• Elixir version: 1.15.7
• OTP version: Erlang/OTP 26 [erts-14.2.5]

I observed the following in my application logs:

** (CaseClauseError) no case clause matching: %Mint.TransportError{reason: :closed}
(finch 0.18.0) lib/finch/http1/conn.ex:52: Finch.HTTP1.Conn.connect/2
(finch 0.18.0) lib/finch/http1/pool.ex:58: anonymous fn/10 in Finch.HTTP1.Pool.request/6
(nimble_pool 1.1.0) lib/nimble_pool.ex:462: NimblePool.checkout!/4
(finch 0.18.0) lib/finch/http1/pool.ex:52: Finch.HTTP1.Pool.request/6
(finch 0.18.0) lib/finch.ex:472: anonymous fn/4 in Finch.request/3
(telemetry 1.2.1) /opt/terrapin-app/deps/telemetry/src/telemetry.erl:321: :telemetry.span/3
(req 0.5.1) lib/req/steps.ex:977: Req.Steps.run_finch_request/3
(req 0.5.1) lib/req/steps.ex:809: Req.Steps.run_finch/4

lib/finch/http1/conn.ex:52:

    case Mint.HTTP.connect(conn.scheme, conn.host, conn.port, conn_opts) do
      {:ok, mint} ->
        Telemetry.stop(:connect, start_time, meta)
        SSL.maybe_log_secrets(conn.scheme, conn_opts, mint)
        {:ok, %{conn | mint: mint}}

      {:error, error} ->
        meta = Map.put(meta, :error, error)
        Telemetry.stop(:connect, start_time, meta)
        {:error, conn, error}
    end

Mint.HTTP1.connect/4 return type should be:

  @spec connect(Types.scheme(), Types.address(), :inet.port_number(), keyword()) ::
          {:ok, t()} | {:error, Types.error()}

Not sure how to replicate given that this for me is a transient issue

[wojtekmach]

~seems like a Finch bug, could you move this issue to Finch repo?~ nevermind

[wojtekmach]

Can you share which URL you hit? Is it over http or https? ipv4, ipv6? Any extra context would really help.

[jozuas]

Looking at my logs further I've discovered another unexpected unwrapped return from Min.HTTP1.connect/4, a nested Mint.TransportError, which seems like a related bug:

** (CaseClauseError) no case clause matching: %Mint.TransportError{reason: %Mint.TransportError{reason: :timeout}}
(finch 0.18.0) lib/finch/http1/conn.ex:52: Finch.HTTP1.Conn.connect/2
(finch 0.18.0) lib/finch/http1/pool.ex:58: anonymous fn/10 in Finch.HTTP1.Pool.request/6
(nimble_pool 1.1.0) lib/nimble_pool.ex:462: NimblePool.checkout!/4
(finch 0.18.0) lib/finch/http1/pool.ex:52: Finch.HTTP1.Pool.request/6
(finch 0.18.0) lib/finch.ex:472: anonymous fn/4 in Finch.request/3
(telemetry 1.2.1) /opt/terrapin-app/deps/telemetry/src/telemetry.erl:321: :telemetry.span/3
(req 0.5.0) lib/req/steps.ex:953: Req.Steps.run_finch_request/3
(req 0.5.0) lib/req/steps.ex:785: Req.Steps.run_finch/4

[jozuas]

Can you share which URL you hit? Is it over http or https? ipv4, ipv6? Any extra context would really help.

I am making Req requests through a pool of proxies.

It looks roughly like:

  @spec base_req() :: Req.Request.t()
  def base_req() do
    proxy = Tp.Proxy.rotate()

    Req.new(
      base_url: "https://...",
      connect_options: [
        proxy: {:http, proxy.ip, proxy.port, []},
        proxy_headers: [{"Proxy-Authorization", "Basic: #{Base.encode64(proxy.userpass)}"}]
      ]
    )
  end

We have around 10 proxies. They are quite unreliable. We get intermittent HTTP 407, :timeout, :ehostunreach, etc. from the proxies.

The connection path is over IPv4, and looks like this:

machine in my control -> http -> proxy -> https -> url

[whatyouhide]

%Mint.TransportError{reason: %Mint.TransportError{reason: :timeout}}

This is definitely an error yeah. I'll try to take a look.