Graceful shutdown doesn't work if metrics or health service is enabled

[sixdayz]

It works for this config:

static:
  dir: "../public"
  forbid: [".php", ".htaccess"]
rpc:
  enable: true
  listen: tcp://127.0.0.1:6001
http:
  address: 0.0.0.0:80
  workers:
    command: "php ../roadrunner/worker.php"
    relay: "tcp://127.0.0.1:10000"

But it doesn't work when I set up another config:

static:
  dir: "../public"
  forbid: [".php", ".htaccess"]
health:
  address: 127.0.0.1:2114
metrics:
  address: 0.0.0.0:2112
rpc:
  enable: true
  listen: tcp://127.0.0.1:6001
http:
  address: 0.0.0.0:80
  workers:
    command: "php ../roadrunner/worker.php"
    relay: "tcp://127.0.0.1:10000"

I tried to remove health and metrics separately. It didn't help. Graceful shutdown doesn't work with any of these sections enabled.

[wolfy-j]

Can you define "Graceful shutdown doesn't work"? Can you provide a minimum reproducible version of the setup? What is the OS?

[sixdayz]

I run my symfony 4 project in docker container based on php:7.3-fpm-alpine with RoadRunner Version: 1.5.0, 2019-10-12T13:06:34+0300.

And I am using this command to run:

rr serve -d -v
      -c roadrunner/config.yaml
      -o http.workers.pool.numWorkers=1
      -o http.workers.pool.maxJobs=1

What information I can provide else?

[wolfy-j]

Can you define "Graceful shutdown doesn't work"? It stops PHP scripts by killing them?

[sixdayz]

Ops!! I closed this issue accidentally. Yes, it stops all workes by killing them without waiting for response.

Example:

search_app    | DEBU[0000] [headers]: disabled
search_app    | DEBU[0000] [rpc]: started
search_app    | DEBU[0000] [http]: started 
search_app    | DEBU[0000] [metrics]: started
search_app    | DEBU[0000] [health]: started
search_app    | DEBU[0009] [rpc]: stopped
search_app    | DEBU[0009] [http]: stopped 
search_app    | DEBU[0009] [metrics]: stopped 
search_app    | DEBU[0009] [health]: stopped
search_app    | ERRO[0009] [health]: http: Server closed
search_app    | Error: [health]: http: Server closed
search_app exited with code 1

And example when it stops workers correctly:

search_app    | DEBU[0000] [metrics]: disabled
search_app    | DEBU[0000] [headers]: disabled 
search_app    | DEBU[0000] [health]: disabled 
search_app    | DEBU[0000] [rpc]: started 
search_app    | DEBU[0000] [http]: started 
search_app    | DEBU[0006] [rpc]: stopped
search_app    | DEBU[0006] [http]: stopped 
search_app    | INFO[0010] 172.25.0.1 {6.243s} 200 GET http://localhost/...?query=&limit=3&with_debug=true
search_app    | ERRO[0010] [http]: http: Server closed
search_app    | Error: [http]: http: Server closed  

[sixdayz]

Looks like it depends on order of stopping services.

[wolfy-j]

RR should wait for all the services to stop. I need to check this issue.

[wolfy-j]

I've managed to reproduce it.

[wolfy-j]

OK, the issue is clear. In case of stop sequence RR will wait for all services to stop gracefully, UNLESS, there is an error. HTTP sometimes sends "Server closed" error which caused the stop sequence to halt. Going to fix it.

[wolfy-j]

Fixed in 1.5.1