search

CrowdHailer / Ace

HTTP web server and client, supports http1 and http2
https://hex.pm/packages/ace
MIT License
305 stars 26 forks source link

how to configure ACE for large file downloads (under http2) #153

Closed CharlesOkwuagwu closed 4 years ago

CharlesOkwuagwu commented 4 years ago

Hi,

I am able to download large files easily with ACE http (cleartext: true) With https download of small files works fine:

iex> [2020-06-25 16:45:23.787] SERVER (port: 88) sent: SETTINGS()
iex> [2020-06-25 16:45:23.790] SERVER (port: 88) received: SETTINGS(header_table_size: 65536, max_concurrent_streams: 1000, initial_window_size: 6291456, max_header_list_size: 262144)
iex> [2020-06-25 16:45:23.793] SERVER (port: 88) received: WINDOWN_UPDATE(stream_id: 0, increment: 15663105)
iex> [2020-06-25 16:45:23.793] SERVER (port: 88) sent: SETTINGS(ack: true)
iex> [2020-06-25 16:45:23.796] SERVER (port: 88) received: HEADERS(stream_id: 1, end_headers: true, end_stream: true)
iex> [2020-06-25 16:45:23.808] SERVER (port: 88) received: SETTINGS(ack: true)
iex> [2020-06-25 16:45:23.810] SERVER (port: 88) sent: HEADERS(stream_id: 1, end_headers: true, end_stream: false)
iex> [2020-06-25 16:45:23.816] SERVER (port: 88) sent: DATA(stream_id: 1, end_stream: true, data: "<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n<!--Designe"...)
iex> [2020-06-25 16:45:23.821] Stopping stream 1 due to :normal
iex> [2020-06-25 16:45:23.824] SERVER (port: 88) sent: RST_STREAM(stream_id: 1, error: internal_error)
iex> [2020-06-25 16:45:24.022] SERVER (port: 88) sent: SETTINGS()
iex> [2020-06-25 16:46:52.971] TLS :server: In state :hello at tls_record.erl:539 generated SERVER ALERT: Fatal - Unexpected Message
 - {:unsupported_record_type, 71}

for large files, it fails:

iex> [2020-06-25 16:47:19.018] SERVER (port: 88) received: HEADERS(stream_id: 3, end_headers: true, end_stream: true)
iex> [2020-06-25 16:47:19.023] SERVER (port: 88) received: PING(ack: false, identifier: <<0, 0, 0, 0, 0, 0, 0, 1>>)
iex> [2020-06-25 16:47:19.023] SERVER (port: 88) sent: PING(ack: true, identifier: <<0, 0, 0, 0, 0, 0, 0, 1>>)
iex> [2020-06-25 16:47:19.104] SERVER (port: 88) sent: HEADERS(stream_id: 3, end_headers: true, end_stream: false)
iex> [2020-06-25 16:47:19.104] SERVER (port: 88) sent: DATA(stream_id: 3, end_stream: false, data: "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<!--Designe"...)
iex> [2020-06-25 16:47:19.104] SERVER (port: 88) sent: DATA(stream_id: 3, end_stream: false, data: "         <voluntary-contribution>0.00</voluntary-c"...)
iex> [2020-06-25 16:47:19.105] SERVER (port: 88) sent: DATA(stream_id: 3, end_stream: false, data: "rial-no>\n          <pin>PEN100307007311</pin>\n    "...)
iex> [2020-06-25 16:47:19.105] SERVER (port: 88) sent: DATA(stream_id: 3, end_stream: false, data: "        <total>5400.00</total>\n        </data>\n   "...)
iex> [2020-06-25 16:47:19.105] SERVER (port: 88) sent: DATA(stream_id: 3, end_stream: false, data: "ntribution>2000.00</employee-contribution>\n       "...)
...
...
[2020-06-25 16:47:19.218] SERVER (port: 88) sent: DATA(stream_id: 3, end_stream: false, data: "al>6338.00</total>\n        </data>\n        <data>\n"...)
[2020-06-25 16:47:19.218] SERVER (port: 88) sent: DATA(stream_id: 3, end_stream: false, data: "    <data>\n          <serial-no>17731</serial-no>\n"...)
[2020-06-25 16:47:19.218] SERVER (port: 88) sent: DATA(stream_id: 3, end_stream: false, data: ">17780</serial-no>\n          <pin>PEN100264756211<"...)
[2020-06-25 16:47:19.233] Stopping stream 3 due to :normal
[2020-06-25 16:47:19.234] SERVER (port: 88) sent: RST_STREAM(stream_id: 3, error: internal_error)

Please what can we adjust to permit larger file downloads with ACE https?

With CURL, for the same resource i get:

C:\Projects\dl                                                                                                                
λ curl -v -k https://localhost:88/file/test.xml                                                                               
*   Trying ::1:88...                                                                                                          
* TCP_NODELAY set                                                                                                             
*   Trying 127.0.0.1:88...                                                                                                    
* TCP_NODELAY set                                                                                                             
* Connected to localhost (127.0.0.1) port 88 (#0)                                                                             
* ALPN, offering h2                                                                                                           
* ALPN, offering http/1.1                                                                                                     
* TLSv1.3 (OUT), TLS handshake, Client hello (1):                                                                             
* TLSv1.3 (IN), TLS handshake, Server hello (2):                                                                              
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):                                                                      
* TLSv1.3 (IN), TLS handshake, Certificate (11):                                                                              
* TLSv1.3 (IN), TLS handshake, CERT verify (15):                                                                              
* TLSv1.3 (IN), TLS handshake, Finished (20):                                                                                 
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):                                                                   
* TLSv1.3 (OUT), TLS handshake, Finished (20):                                                                                
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384                                                                       
* ALPN, server accepted to use h2                                                                                             
* Server certificate:                                                                                                         
*  subject: C=NG; ST=Lagos; L=.....................................                        
*  start date: Apr 14 00:00:00 2020 GMT                                                                                       
*  expire date: Dec  4 12:00:00 2020 GMT                                                                                      
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA                                                            
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.                             
* Using HTTP2, server supports multi-use                                                                                      
* Connection state changed (HTTP/2 confirmed)                                                                                 
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0                                              
* Using Stream ID: 1 (easy handle 0x271735f9130)                                                                              
> GET /file/test.xml HTTP/2                                                                                                   
> Host: localhost:88                                                                                                          
> User-Agent: curl/7.65.0                                                                                                     
> Accept: */*                                                                                                                 
>                                                                                                                             
* Connection state changed (MAX_CONCURRENT_STREAMS == 4294967295)!                                                            
< HTTP/2 200                                                                                                                  
< content-type: text/xml                                                                                                      
< content-disposition: attachment; filename="test.xml"                                                                        
< content-length: 198134182           
...
...
           <total>2108.00</total>
        </data>* transfer closed with 193041621 bytes remaining to read
* stopped the pause stream!
* Connection #0 to host localhost left intact
curl: (18) transfer closed with 193041621 bytes remaining to read

C:\Projects\dl
λ
CrowdHailer commented 4 years ago

I'm not going to have time to look into this now. If you need help I would perhaps make a comment here, https://elixirforum.com/t/help-maintaining-elixir-http-2-server-nearly-at-1-0-release/30279/2

CharlesOkwuagwu commented 4 years ago

@CrowdHailer Thanks man.

CharlesOkwuagwu commented 4 years ago

@CrowdHailer Resolved in this PR #https://github.com/CrowdHailer/Ace/pull/154