docker-archive / classicswarm

Swarm Classic: a container clustering system. Not to be confused with Docker Swarm which is at https://github.com/docker/swarmkit
Apache License 2.0
5.75k stars 1.08k forks source link

docker stack rm - SIGTERM not sent to the running process #2897

Closed aphasia0 closed 4 years ago

aphasia0 commented 6 years ago

Hello,

I need to execute a custom shutdown procedure for my containers. To handle this I've added a trap in the entry point script:

#!/bin/bash
trap 'myCustomStopScripr.sh' SIGTERM
...

This works very well if I use:

docker stop myContainerId

but, using swarm I'm not able to catch any signal from the container, so my stops will be never graceful. I've tried:

docker stack rm myStackName
docker service rm myServiceName
docker service scale myServiceName=0

All the above commands kills my container with SIGKILL I think (waiting the default grace period of 10 seconds). It seems like swarm behind the scenes uses:

docker container rm -f myContainerId

EDIT: My docker version: Version: 1.13.1 API version: 1.26 (minimum version 1.12)

EDIT2: I definitely found where the issue is. I receive the SIGTERM into the container but the shutdown procedure fails. The reason is that I cannot use neither ServiceName nor any alias. It seems like the overlay network is detached and then the signal is sent to the container.

Can you help me? Is this a bug or I'm doing something basically wrong?

Thank you.

bohendo commented 5 years ago

Have the same problem, someone in another conversation pointed out that the SIGTERM is getting sent but it's followed almost immediately by SIGKILL. They said that the stop_grade_period option fixed it for them.

Modifying the stop_grace_period option.. didn't change anything for me though. 🙁

$ docker version
Client: Docker Engine - Community
 Version:           18.09.1
 API version:       1.39
 Go version:        go1.10.6
 Git commit:        4c52b90
 Built:             Wed Jan  9 19:33:12 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.1
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.6
  Git commit:       4c52b90
  Built:            Wed Jan  9 19:41:49 2019
  OS/Arch:          linux/amd64
  Experimental:     true
bohendo commented 5 years ago

Aaaand looks like my problem was that I had processes running in the background & docker stop signals are only sent to PID 1. I was confused by which process was PID 1 and found a solution to pass stop signals to child processes.

Now that the proper processes are getting the proper signals, my stuff is working w/out needing to mess w the stop_grace_period option.

aphasia0 commented 5 years ago

I definitely solved not using swarm and overlay network at all. I’ve used nomad+consul instead, it’s more production ready and well supported imho.