search

tempesta-tech / tempesta

All-in-one solution for high performance web content delivery and advanced protection against DDoS and web attacks
https://tempesta-tech.com/
GNU General Public License v2.0
620 stars 103 forks source link

Chunked transfer encoding: PURGE+GET request connection is not closed #1692

Open b3b opened 2 years ago

b3b commented 2 years ago

Scope

For PURGE requests with the X-Tempesta-Cache: get header set:

WordPress https://github.com/tempesta-tech/tempesta-test/issues/290 home page purging example:

# curl --verbose --request PURGE --max-time 16 172.17.0.1 -H 'X-Tempesta-Cache: get'
*   Trying 172.17.0.1:80...
* Connected to 172.17.0.1 (172.17.0.1) port 80 (#0)
> PURGE / HTTP/1.1
> Host: 172.17.0.1
> User-Agent: curl/7.83.1
> Accept: */*
> X-Tempesta-Cache: get
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 16 Sep 2022 10:25:42 GMT
< X-Powered-By: PHP/7.4.30
< Link: <http://127.0.0.1/index.php?rest_route=/>; rel="https://api.w.org/"
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0via: 1.1 tempesta_fw (Tempesta FW pre-0.7.0)
< Server: Tempesta FW/pre-0.7.0
< 
* Operation timed out after 16000 milliseconds with 0 bytes received
* Closing connection 0
curl: (28) Operation timed out after 16000 milliseconds with 0 bytes received

Raw response from WordPress:

HTTP/1.1 200 OK
Date: Fri, 16 Sep 2022 10:25:42 GMT
Server: Apache/2.4.54 (Debian)
X-Powered-By: PHP/7.4.30
Link: <http://127.0.0.1/index.php?rest_route=/>; rel="https://api.w.org/"
Vary: Accept-Encoding
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
\r\n
142b0\r\n
<!DOCTYPE html>
...
</html>
\r\n
0\r\n
\r\n

Testing

Test to reproduce: cache.test_purge.TestPurgeGetWithTransferEncoding

krizhanovsky commented 2 years ago

Why Tempesta should close the connection? There is no Connection header in the curl request - does RFC requires to close connection by default?

b3b commented 2 years ago

If connection is not closed, then chunked body is expected by a client, but body is cutted off by Tempesta. And even if Connection: close is set by a client, header is replaced with Connection: keep-alive by Tempesta. +Content-Length header has broken value instead of 0.

Session with the Connection: close set by a client:

# curl --verbose --max-time 16 172.17.0.1 -X PURGE -H 'X-Tempesta-Cache: get' -H 'Connection: close'
* Trying 172.17.0.1:80...
* Connected to 172.17.0.1 (172.17.0.1) port 80 (#0)
> PURGE / HTTP/1.1
> Host: 172.17.0.1
> User-Agent: curl/7.83.1
> Accept: */*
> X-Tempesta-Cache: get
> Connection: close
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 16 Sep 2022 13:09:36 GMT
< X-Powered-By: PHP/7.4.30
< Link: <http://127.0.0.1/index.php?rest_route=/>; rel="https://api.w.org/"
< Vary: Accept-Encoding
< Connection: close
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0via: 1.1 tempesta_fw (Tempesta FW pre-0.7.0)
< Server: Tempesta FW/pre-0.7.0
< 
* transfer closed with outstanding read data remaining
* Closing connection 0
curl: (18) transfer closed with outstanding read data remaining
b3b commented 2 years ago

Test added to check that Connection: close header is not passed to backend: cache.test_purge.TestPurgeGetWithTransferEncoding

krizhanovsky commented 2 years ago

It seems the problem is that the backend sends Transfer-Encoding: chunked and when Tempesta FW cuts off the response for X-Tempesta-Cache: get if doesn't cut off Transfer-Encoding.