search

hjr3 / weldr

A HTTP 1.1 proxy written in Rust using tokio.
Apache License 2.0
217 stars 20 forks source link

handle response body when the head is sent before the body #35

Closed yanns closed 7 years ago

yanns commented 7 years ago

When getting an asset directly on the server (port 9000)

wget http://localhost:9000/assets/javascripts/jquery-1.7.1.min.js
--2016-11-29 10:39:19--  http://localhost:9000/assets/javascripts/jquery-1.7.1.min.js
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:9000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 93868 (92K) [application/javascript]
Saving to: ‘jquery-1.7.1.min.js’

jquery-1.7.1.min.js                                                 100%[===================================================================================================================================================================>]  91.67K  --.-KB/s    in 0s

2016-11-29 10:39:19 (220 MB/s) - ‘jquery-1.7.1.min.js’ saved [93868/93868]
12  18.907855   ::1 ::1 TCP 88  57009→9000 [SYN] Seq=0 Win=65535 Len=0 MSS=16324 WS=32 TSval=227255021 TSecr=0 SACK_PERM=1
13  18.907918   ::1 ::1 TCP 88  9000→57009 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=16324 WS=32 TSval=227255021 TSecr=227255021 SACK_PERM=1
14  18.907928   ::1 ::1 TCP 76  57009→9000 [ACK] Seq=1 Ack=1 Win=407776 Len=0 TSval=227255021 TSecr=227255021
15  18.907938   ::1 ::1 TCP 76  [TCP Window Update] 9000→57009 [ACK] Seq=1 Ack=1 Win=407776 Len=0 TSval=227255021 TSecr=227255021
16  18.907960   ::1 ::1 HTTP    256 GET /assets/javascripts/jquery-1.7.1.min.js HTTP/1.1 

GET /assets/javascripts/jquery-1.7.1.min.js HTTP/1.1
User-Agent: Wget/1.18 (darwin15.5.0)
Accept: */*
Accept-Encoding: identity
Host: localhost:9000
Connection: Keep-Alive

17  18.907986   ::1 ::1 TCP 76  9000→57009 [ACK] Seq=1 Ack=181 Win=407616 Len=0 TSval=227255021 TSecr=227255021
18  18.914313   ::1 ::1 TCP 329 [TCP segment of a reassembled PDU]

HTTP/1.1 200 OK
ETag: "efab29516dc38c40118a01a7888ffd6b30e8b971"
Cache-Control: no-cache
Last-Modified: Tue, 08 Mar 2016 10:25:02 GMT
Content-Length: 93868
Content-Type: application/javascript; charset=utf-8
Date: Tue, 29 Nov 2016 09:39:19 GMT

19  18.914349   ::1 ::1 TCP 76  57009→9000 [ACK] Seq=181 Ack=254 Win=407520 Len=0 TSval=227255027 TSecr=227255027
20  18.916082   ::1 ::1 TCP 8268    [TCP segment of a reassembled PDU]

/*! jQuery v1.7.1 jquery.com | jquery.org/license */
[...body follows]

With alacrity running on port 9001 (with branch https://github.com/yanns/alacrity/tree/buffer_body_when_incomplete):

wget http://localhost:9001/assets/javascripts/jquery-1.7.1.min.js
--2016-11-29 10:57:14--  http://localhost:9001/assets/javascripts/jquery-1.7.1.min.js
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:9001... failed: Connection refused.
Connecting to localhost|127.0.0.1|:9001... connected.
HTTP request sent, awaiting response...
3   0.000133    127.0.0.1   127.0.0.1   TCP 68  57435→9001 [SYN] Seq=0 Win=65535 Len=0 MSS=16344 WS=32 TSval=229077305 TSecr=0 SACK_PERM=1
4   0.000178    127.0.0.1   127.0.0.1   TCP 68  9001→57435 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=16344 WS=32 TSval=229077305 TSecr=229077305 SACK_PERM=1
5   0.000187    127.0.0.1   127.0.0.1   TCP 56  57435→9001 [ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=229077305 TSecr=229077305
6   0.000209    127.0.0.1   127.0.0.1   TCP 56  [TCP Window Update] 9001→57435 [ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=229077305 TSecr=229077305
7   0.000235    127.0.0.1   127.0.0.1   HTTP    236 GET /assets/javascripts/jquery-1.7.1.min.js HTTP/1.1 

GET /assets/javascripts/jquery-1.7.1.min.js HTTP/1.1
User-Agent: Wget/1.18 (darwin15.5.0)
Accept: */*
Accept-Encoding: identity
Host: localhost:9001
Connection: Keep-Alive

8   0.000264    127.0.0.1   127.0.0.1   TCP 56  9001→57435 [ACK] Seq=1 Ack=181 Win=408096 Len=0 TSval=229077305 TSecr=229077305
9   0.000880    127.0.0.1   127.0.0.1   TCP 68  57436→9000 [SYN] Seq=0 Win=65535 Len=0 MSS=16344 WS=32 TSval=229077305 TSecr=0 SACK_PERM=1
11  0.000923    127.0.0.1   127.0.0.1   TCP 56  57436→9000 [ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=229077305 TSecr=229077305
12  0.000930    127.0.0.1   127.0.0.1   TCP 56  [TCP Window Update] 9000→57436 [ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=229077305 TSecr=229077305
13  0.001382    127.0.0.1   127.0.0.1   HTTP    236 GET /assets/javascripts/jquery-1.7.1.min.js HTTP/1.1 

GET /assets/javascripts/jquery-1.7.1.min.js HTTP/1.1
User-Agent: Wget/1.18 (darwin15.5.0)
Accept: */*
Accept-Encoding: identity
Host: localhost:9001
Connection: Keep-Alive

14  0.001399    127.0.0.1   127.0.0.1   TCP 56  9000→57436 [ACK] Seq=1 Ack=181 Win=408096 Len=0 TSval=229077306 TSecr=229077306
15  0.006116    127.0.0.1   127.0.0.1   TCP 309 [TCP segment of a reassembled PDU]

HTTP/1.1 200 OK
ETag: "efab29516dc38c40118a01a7888ffd6b30e8b971"
Cache-Control: no-cache
Last-Modified: Tue, 08 Mar 2016 10:25:02 GMT
Content-Length: 93868
Content-Type: application/javascript; charset=utf-8
Date: Tue, 29 Nov 2016 10:10:04 GMT

16  0.006155    127.0.0.1   127.0.0.1   TCP 56  57436→9000 [ACK] Seq=181 Ack=254 Win=408032 Len=0 TSval=229077310 TSecr=229077310
17  0.008169    127.0.0.1   127.0.0.1   TCP 8248    [TCP segment of a reassembled PDU]

/*! jQuery v1.7.1 jquery.com | jquery.org/license */
[...body follows]

39  0.008518    127.0.0.1   127.0.0.1   HTTP    3812    HTTP/1.1 200 OK  (application/javascript)

[end of body]

40  0.008530    127.0.0.1   127.0.0.1   TCP 56  57436→9000 [ACK] Seq=181 Ack=94122 Win=314176 Len=0 TSval=229077312 TSecr=229077312

So we can see that alacrity is buffering the body as expected. The server sends the whole javascript. But alacrity never sent the http response to the client.

hjr3 commented 7 years ago

The tokio buffer is 8096 bytes. This means we have to do multiple reads from the backend and writes to the frontend to serve a large file. I have this mostly done and can post where I am at later today. Currently, I am working through the FramedProxy implementation and how it interacts with tokio-proto. I can post a WIP PR later today so you can take a look.