elixir-tesla / tesla

The flexible HTTP client library for Elixir, with support for middleware and multiple adapters.
MIT License
2k stars 340 forks source link

Mint adapter cannot upload more than 65535 bytes on HTTP/2 #394

Open jayjun opened 4 years ago

jayjun commented 4 years ago

To reproduce,

client = Tesla.client([], Tesla.Adapter.Mint)
Tesla.post(client, "https://httpbin.org/post", String.duplicate("0", 65536))

returns :exceeds_window_size error.

%Mint.HTTPError{
  module: Mint.HTTP2,
  reason: {:exceeds_window_size, :connection, 65535}
}
Full error ```elixir {:error, %Mint.HTTP2{ buffer: <<0, 0, 4, 8, 0, 0, 0, 0, 0, 127, 255, 0, 0>>, client_settings: %{ enable_push: true, max_concurrent_streams: 100, max_frame_size: 16384 }, client_settings_queue: {[[]], []}, decode_table: %Mint.HTTP2.HPACK.Table{ entries: [], length: 0, max_table_size: 4096, size: 0 }, encode_table: %Mint.HTTP2.HPACK.Table{ entries: [], length: 0, max_table_size: 4096, size: 0 }, headers_being_processed: nil, hostname: "httpbin.org", mode: :active, next_stream_id: 5, open_client_stream_count: 1, open_server_stream_count: 0, ping_queue: {[], []}, port: 443, private: %{}, ref_to_stream_id: %{#Reference<0.4203460067.3989045250.44202> => 3}, scheme: "https", server_settings: %{ enable_push: true, initial_window_size: 65536, max_concurrent_streams: 128, max_frame_size: 16777215, max_header_list_size: :infinity }, socket: {:sslsocket, {:gen_tcp, #Port<0.2811>, :tls_connection, :undefined}, [#PID<0.1699.0>, #PID<0.1698.0>]}, state: :open, streams: %{ 3 => %{ id: 3, received_first_headers?: false, ref: #Reference<0.4203460067.3989045250.44202>, state: :open, window_size: 65536 } }, transport: Mint.Core.Transport.SSL, window_size: 65535 }, %Mint.HTTPError{ module: Mint.HTTP2, reason: {:exceeds_window_size, :connection, 65535} }} ```

That is because window_size: 65535 is set initially, according to standard, but is not updated when the server sends WINDOW_UPDATE frames.

Each time Tesla sends data using Mint.HTTP.stream_request_body, the window size is decremented by data’s size. When the window size gets low, the server sends WINDOW_UPDATE frames to increment the connection’s window size. Calling Mint.HTTP.stream to read responses will automatically do this.

https://github.com/teamon/tesla/blob/754d4a6ffe41ff508fc7fa460508a40e965b2960/lib/tesla/adapter/mint.ex#L178-L187

However, the adapter repeatedly calls Mint.HTTP.stream_request_body without calling Mint.HTTP.stream so the window size hits zero and Mint errors.

teamon commented 4 years ago

Hi @jayjun, thanks for reporting the issue. Would you be able to provide the fix as a PR?

jayjun commented 4 years ago

Unfortunately not at the moment because I don’t have the bandwidth to learn Tesla internals, Mint and HTTP/2 enough to do a good job.

slapers commented 2 years ago

If someone hits this error until this is solved, as a temporary solution you can force the connection to be http1 by adding some options for the adapter:

Tesla.post(client, url, content, [opts: [adapter: [protocols: [:http1]]]])
sheharyarn commented 1 year ago

Bumping this since this is a serious issue!

Thank you for all the work on Tesla!

yordis commented 1 year ago

@sheharyarn do you have some bandwidth to tackle the issue?

sheharyarn commented 1 year ago

@yordis I can definitely try, but would need someone to push me in the right direction. While I have a good amount of experience with Tesla in general, I don't have enough experience with Mint's internals.