dallyger commented 1 day ago

Summary:

A request with the header "Transfer-Encoding: chunked" will cause Caddy to spawn a php_fastcgi process, which hangs forever.

After upgrading to Shopware 6.6 (using shopware/docker), our shop would timeout every couple hours/days due to bot traffic until manually restarted. Our old setup with nginx did not have that issue.

Temporary solution

I've added the following entry to our Varnish config to block such requests:

 sub vcl_recv {

+    if (req.http.Transfer-Encoding == "chunked") {
+        return(synth(411, "Length Required"));
+    }

     # ...
 }

Another option would be to set request_terminate_timeout = 120s in php.conf, so hanging requests are automatically terminated after two minutes.

How to reproduce:

Setup a Shopware instance:

composer create-project shopware/production:6.6.6.1 .
composer req shopware/docker

Start a docker container from an image build by using that project by adding the following to compose.yaml:

services:
  web:
    build:
      context: .
      dockerfile: docker/Dockerfile
    depends_on:
      - database
    env_file:
      - .env
    environment:
      DATABASE_URL: mysql://${MYSQL_USER:-shopware}:${MYSQL_PASSWORD:-!ChangeMe!}@database/${MYSQL_DATABASE:-shopware}
    ports:
      - 8000:8000
    volumes:
      - .:/var/www/html

And finally, send a broken request using curl:

curl --insecure -X POST http://127.0.0.1:8000/ \
    --header "Content-Type: application/json" \
    --header "Transfer-Encoding: chunked" \
    --http1.1 \
    --data '{}' -vvv

Request will hang and no body is returned:

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1:8000...
* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
> POST / HTTP/1.1
> Host: 127.0.0.1:8000
> User-Agent: curl/7.81.0
> Accept: */*
> Content-Type: application/json
> Transfer-Encoding: chunked
>

Run the following command to observe an additional, forever open socket:

docker compose exec web netstat -alWx

Repeat this 5 times (default value of pm.max_children) and the shop is unresponsive and any further requests will hang forever too, even if they're not chunked. Or a 502 timeout occurs, if behind a reverse proxy.

shopware / docker

Transfer-Encoding: chunked causing denial of service #107

Summary:

Temporary solution

How to reproduce:

See also