search

uNetworking / uWebSockets

Simple, secure & standards compliant web server for the most demanding of applications
Apache License 2.0
17.42k stars 1.76k forks source link

Http request error 505 when the first line of the request is fragmented. #1776

Closed kindablissy closed 1 month ago

kindablissy commented 2 months ago

If the first line of http request somehow misses to go in the same packet as [method] [path] [version]\r\n but instead gets something like [method] and [path] [version]\r\n it will return with version not supported error. Is this expected ?

uNetworkingAB commented 2 months ago

Must be a regression from when more detailed errors were added because this is not the old behavior

uNetworkingAB commented 2 months ago

Needs unit test and fix

slavslavov commented 2 months ago

I am also seeing this error when trying to serve a Qt application built for Webassembly. I am attaching the files which are served if this will help for debugging. If I serve the files locally, all works. But my real setup is on a remote server with HAProxy terminating the TLS certificate and then forwarding to uWebsockets server. Webassembly application will require these two headers to be set

                res->writeHeader("Cross-Origin-Embedder-Policy", "require-corp");
                res->writeHeader("Cross-Origin-Opener-Policy", "same-origin");

gallery.zip

uNetworkingAB commented 2 months ago

There is no debugging needed it happens here https://github.com/uNetworking/uWebSockets/blob/9cca5d68e0ac7153d8250445c076a9ffc3d58dae/src/HttpParser.h#L370

It needs to check whether it reached end of data and if so, just return 0 instead of setting error. Then it will keep accepting more data instead of immediately closing with error.

slavslavov commented 2 months ago

If I comment out that line I don't see the error any more so as you suggest it is probably an easy fix.

But I was curious why I see this error when I am using HAProxy. I noticed the error when I am using the Proxy Protocol, and uWebsockets is compiled to support it. If I disable HAProxy sending the proxy protocol header I don't see the error any more.

I did use tcpdump and wireshark to capture the traffic between HAProxy and uWebsockets. I noticed that HAproxy checks the connection every two seconds and also sends the protocol header. And I can see the error sometimes returned in these checks. Here is part of the dump and wireshark capture is attached:

capture.zip

12:14:05.971392 IP localhost.10046 > localhost.65012: Flags [S], seq 1227822058, win 65495, options [mss 65495,sackOK,TS val 3656833800 ecr 0,nop,wscale 7], length 0
    0x0000:  4500 003c fd7c 4000 4006 da3c 7f00 6501  E..<.|@.@..<..e.
    0x0010:  7f00 0001 273e fdf4 492f 13ea 0000 0000  ....'>..I/......
    0x0020:  a002 ffd7 6331 0000 0204 ffd7 0402 080a  ....c1..........
    0x0030:  d9f6 db08 0000 0000 0103 0307            ............
12:14:05.971414 IP localhost.65012 > localhost.10046: Flags [S.], seq 1888127914, ack 1227822059, win 65483, options [mss 65495,sackOK,TS val 4167233272 ecr 3656833800,nop,wscale 7], length 0
    0x0000:  4500 003c 0000 4000 4006 d7b9 7f00 0001  E..<..@.@.......
    0x0010:  7f00 6501 fdf4 273e 708a 8baa 492f 13eb  ..e...'>p...I/..
    0x0020:  a012 ffcb 6331 0000 0204 ffd7 0402 080a  ....c1..........
    0x0030:  f862 eef8 d9f6 db08 0103 0307            .b..........
12:14:05.971487 IP localhost.10046 > localhost.65012: Flags [P.], seq 1:17, ack 1, win 512, options [nop,nop,TS val 3656833800 ecr 4167233272], length 16
    0x0000:  4500 0044 fd7d 4000 4006 da33 7f00 6501  E..D.}@.@..3..e.
    0x0010:  7f00 0001 273e fdf4 492f 13eb 708a 8bab  ....'>..I/..p...
    0x0020:  8018 0200 6339 0000 0101 080a d9f6 db08  ....c9..........
    0x0030:  f862 eef8 0d0a 0d0a 000d 0a51 5549 540a  .b.........QUIT.
    0x0040:  2000 0000                                ....
12:14:05.971526 IP localhost.65012 > localhost.10046: Flags [.], ack 17, win 512, options [nop,nop,TS val 4167233272 ecr 3656833800], length 0
    0x0000:  4500 0034 e152 4000 4006 f66e 7f00 0001  E..4.R@.@..n....
    0x0010:  7f00 6501 fdf4 273e 708a 8bab 492f 13fb  ..e...'>p...I/..
    0x0020:  8010 0200 6329 0000 0101 080a f862 eef8  ....c).......b..
    0x0030:  d9f6 db08                                ....
12:14:05.971559 IP localhost.10046 > localhost.65012: Flags [R.], seq 17, ack 1, win 512, options [nop,nop,TS val 3656833800 ecr 4167233272], length 0
    0x0000:  4500 0034 fd7e 4000 4006 da42 7f00 6501  E..4.~@.@..B..e.
    0x0010:  7f00 0001 273e fdf4 492f 13fb 708a 8bab  ....'>..I/..p...
    0x0020:  8014 0200 6329 0000 0101 080a d9f6 db08  ....c)..........
    0x0030:  f862 eef8                                .b..

12:14:07.973053 IP localhost.10047 > localhost.65012: Flags [S], seq 2806089025, win 65495, options [mss 65495,sackOK,TS val 3656835802 ecr 0,nop,wscale 7], length 0
    0x0000:  4500 003c 3487 4000 4006 a332 7f00 6501  E..<4.@.@..2..e.
    0x0010:  7f00 0001 273f fdf4 a741 8541 0000 0000  ....'?...A.A....
    0x0020:  a002 ffd7 6331 0000 0204 ffd7 0402 080a  ....c1..........
    0x0030:  d9f6 e2da 0000 0000 0103 0307            ............
12:14:07.973078 IP localhost.65012 > localhost.10047: Flags [S.], seq 587897328, ack 2806089026, win 65483, options [mss 65495,sackOK,TS val 4167235274 ecr 3656835802,nop,wscale 7], length 0
    0x0000:  4500 003c 0000 4000 4006 d7b9 7f00 0001  E..<..@.@.......
    0x0010:  7f00 6501 fdf4 273f 230a 99f0 a741 8542  ..e...'?#....A.B
    0x0020:  a012 ffcb 6331 0000 0204 ffd7 0402 080a  ....c1..........
    0x0030:  f862 f6ca d9f6 e2da 0103 0307            .b..........
12:14:07.973158 IP localhost.10047 > localhost.65012: Flags [P.], seq 1:17, ack 1, win 512, options [nop,nop,TS val 3656835802 ecr 4167235274], length 16
    0x0000:  4500 0044 3488 4000 4006 a329 7f00 6501  E..D4.@.@..)..e.
    0x0010:  7f00 0001 273f fdf4 a741 8542 230a 99f1  ....'?...A.B#...
    0x0020:  8018 0200 6339 0000 0101 080a d9f6 e2da  ....c9..........
    0x0030:  f862 f6ca 0d0a 0d0a 000d 0a51 5549 540a  .b.........QUIT.
    0x0040:  2000 0000                                ....
12:14:07.973199 IP localhost.65012 > localhost.10047: Flags [.], ack 17, win 512, options [nop,nop,TS val 4167235274 ecr 3656835802], length 0
    0x0000:  4500 0034 38df 4000 4006 9ee2 7f00 0001  E..48.@.@.......
    0x0010:  7f00 6501 fdf4 273f 230a 99f1 a741 8552  ..e...'?#....A.R
    0x0020:  8010 0200 6329 0000 0101 080a f862 f6ca  ....c).......b..
    0x0030:  d9f6 e2da                                ....
12:14:07.973529 IP localhost.65012 > localhost.10047: Flags [P.], seq 1:63, ack 17, win 512, options [nop,nop,TS val 4167235274 ecr 3656835802], length 62
    0x0000:  4500 0072 38e0 4000 4006 9ea3 7f00 0001  E..r8.@.@.......
    0x0010:  7f00 6501 fdf4 273f 230a 99f1 a741 8552  ..e...'?#....A.R
    0x0020:  8018 0200 6367 0000 0101 080a f862 f6ca  ....cg.......b..
    0x0030:  d9f6 e2da 4854 5450 2f31 2e31 2035 3035  ....HTTP/1.1.505
    0x0040:  2048 5454 5020 5665 7273 696f 6e20 4e6f  .HTTP.Version.No
    0x0050:  7420 5375 7070 6f72 7465 640d 0a43 6f6e  t.Supported..Con
    0x0060:  6e65 6374 696f 6e3a 2063 6c6f 7365 0d0a  nection:.close..
    0x0070:  0d0a                                     ..
12:14:07.973559 IP localhost.65012 > localhost.10047: Flags [F.], seq 63, ack 17, win 512, options [nop,nop,TS val 4167235274 ecr 3656835802], length 0
    0x0000:  4500 0034 38e1 4000 4006 9ee0 7f00 0001  E..48.@.@.......
    0x0010:  7f00 6501 fdf4 273f 230a 9a2f a741 8552  ..e...'?#../.A.R
    0x0020:  8011 0200 6329 0000 0101 080a f862 f6ca  ....c).......b..
    0x0030:  d9f6 e2da                                ....
12:14:07.973650 IP localhost.10047 > localhost.65012: Flags [F.], seq 17, ack 64, win 512, options [nop,nop,TS val 3656835802 ecr 4167235274], length 0
    0x0000:  4500 0034 3489 4000 4006 a338 7f00 6501  E..44.@.@..8..e.
    0x0010:  7f00 0001 273f fdf4 a741 8552 230a 9a30  ....'?...A.R#..0
    0x0020:  8011 0200 6329 0000 0101 080a d9f6 e2da  ....c)..........
    0x0030:  f862 f6ca                                .b..
12:14:07.973675 IP localhost.65012 > localhost.10047: Flags [.], ack 18, win 512, options [nop,nop,TS val 4167235274 ecr 3656835802], length 0
    0x0000:  4500 0034 38e2 4000 4006 9edf 7f00 0001  E..48.@.@.......
    0x0010:  7f00 6501 fdf4 273f 230a 9a30 a741 8553  ..e...'?#..0.A.S
    0x0020:  8010 0200 6329 0000 0101 080a f862 f6ca  ....c).......b..
    0x0030:  d9f6 e2da                                ....

12:14:09.974951 IP localhost.10048 > localhost.65012: Flags [S], seq 3141492745, win 65495, options [mss 65495,sackOK,TS val 3656837804 ecr 0,nop,wscale 7], length 0
    0x0000:  4500 003c 65ba 4000 4006 71ff 7f00 6501  E..<e.@.@.q...e.
    0x0010:  7f00 0001 2740 fdf4 bb3f 6009 0000 0000  ....'@...?`.....
    0x0020:  a002 ffd7 6331 0000 0204 ffd7 0402 080a  ....c1..........
    0x0030:  d9f6 eaac 0000 0000 0103 0307            ............
12:14:09.974975 IP localhost.65012 > localhost.10048: Flags [S.], seq 54041943, ack 3141492746, win 65483, options [mss 65495,sackOK,TS val 4167237276 ecr 3656837804,nop,wscale 7], length 0
    0x0000:  4500 003c 0000 4000 4006 d7b9 7f00 0001  E..<..@.@.......
    0x0010:  7f00 6501 fdf4 2740 0338 9d57 bb3f 600a  ..e...'@.8.W.?`.
    0x0020:  a012 ffcb 6331 0000 0204 ffd7 0402 080a  ....c1..........
    0x0030:  f862 fe9c d9f6 eaac 0103 0307            .b..........
12:14:09.975048 IP localhost.10048 > localhost.65012: Flags [P.], seq 1:17, ack 1, win 512, options [nop,nop,TS val 3656837804 ecr 4167237276], length 16
    0x0000:  4500 0044 65bb 4000 4006 71f6 7f00 6501  E..De.@.@.q...e.
    0x0010:  7f00 0001 2740 fdf4 bb3f 600a 0338 9d58  ....'@...?`..8.X
    0x0020:  8018 0200 6339 0000 0101 080a d9f6 eaac  ....c9..........
    0x0030:  f862 fe9c 0d0a 0d0a 000d 0a51 5549 540a  .b.........QUIT.
    0x0040:  2000 0000                                ....
12:14:09.975087 IP localhost.65012 > localhost.10048: Flags [.], ack 17, win 512, options [nop,nop,TS val 4167237276 ecr 3656837804], length 0
    0x0000:  4500 0034 b5f5 4000 4006 21cc 7f00 0001  E..4..@.@.!.....
    0x0010:  7f00 6501 fdf4 2740 0338 9d58 bb3f 601a  ..e...'@.8.X.?`.
    0x0020:  8010 0200 6329 0000 0101 080a f862 fe9c  ....c).......b..
    0x0030:  d9f6 eaac                                ....
12:14:09.975128 IP localhost.10048 > localhost.65012: Flags [R.], seq 17, ack 1, win 512, options [nop,nop,TS val 3656837804 ecr 4167237276], length 0
    0x0000:  4500 0034 65bc 4000 4006 7205 7f00 6501  E..4e.@.@.r...e.
    0x0010:  7f00 0001 2740 fdf4 bb3f 601a 0338 9d58  ....'@...?`..8.X
    0x0020:  8014 0200 6329 0000 0101 080a d9f6 eaac  ....c)..........
    0x0030:  f862 fe9c
uNetworkingAB commented 2 months ago

Needs a good test that just takes a simple request, chunks it up into every combination of two segements and asserts same outcome for all. Working on it

uNetworkingAB commented 2 months ago

You mind testing latest commit? It still lacks a test but should probably work now

slavslavov commented 2 months ago

I tested with commit 6b966b24ee2d186fcfd86c3f3d450fa913782c60 and I can't see the error any more. I used the same environment as before - HAProxy terminating TLS and forwarding to uWebsockets with checks and Proxy Protocol V2. I ran tcpdump and wireshark for about 20 minutes, also quite intensively requesting a large .wasm file. All looks good now.

uNetworkingAB commented 2 months ago

Great I just need a unit test for this and will release