DarthSim / overmind

Process manager for Procfile-based applications and tmux
MIT License
2.84k stars 79 forks source link

Docker process not properly ended #57

Closed bswinnerton closed 4 years ago

bswinnerton commented 5 years ago

Hi there!

I'm running into an issue where a docker run process, executed via a script, isn't properly killed by Overmind's SIGINT.

My Procfile reads as follows:

web: script/server
websockify: script/websockify-server

The script/websockify-server reads:

#!/bin/sh

docker run --rm -it -p $WEBSOCKIFY_PORT:80 --name websockify websockify 80 --token-plugin BaseTokenAPI --token-source "http://docker.for.mac.localhost:$PORT/internal/websockify_tokens/%s?secret=$WEBSOCKIFY_TOKENS_SECRET"

When running script/websockify-server, I can properly kill it with ctrl +c:

$ script/websockify-server
WebSocket server settings:
  - Listen on :80
  - No SSL/TLS support (no cert file)
  - proxying from :80 to targets generated by BaseTokenAPI
^CIn exit
$ docker ps --all
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

But when running overmind start and killing with a ctrl +c, the docker container is left running:

$ overmind start
system     | Tmux socket name: overmind-neptune-networks-b_4W9kfW1Ie6_NoeZnFaKO
system     | Tmux session ID: neptune-networks
system     | Listening at /Users/brooks/Sites/neptune-networks/neptune-networks/.overmind.sock
web        | Started with pid 25864...
websockify | Started with pid 25865...
websockify | WebSocket server settings:
websockify |   - Listen on :80
websockify |   - No SSL/TLS support (no cert file)
websockify |   - proxying from :80 to targets generated by BaseTokenAPI
web        | => Booting Puma
web        | => Rails 5.2.3 application starting in development
web        | => Run `rails server -h` for more startup options
web        | Puma starting in single mode...
web        | * Version 3.12.1 (ruby 2.6.4-p104), codename: Llamas in Pajamas
web        | * Min threads: 5, max threads: 5
web        | * Environment: development
web        | * Listening on tcp://localhost:5000
web        | Use Ctrl-C to stop
^Cweb        | Interrupting...
websockify | Interrupting...
websockify |
web        | - Gracefully stopping, waiting for requests to finish
web        | === puma shutdown: 2019-09-07 11:39:15 -0400 ===
web        | - Goodbye!
web        | Exiting
web        | Exited
websockify | Exited
$ docker ps --all
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                           NAMES
1bb9953982c2        websockify          "/opt/websockify/run…"   5 seconds ago       Up 4 seconds        443/tcp, 0.0.0.0:9090->80/tcp   websockify

At first I thought this may be related to docker running as PID 1 and needing a --init passed to docker run to properly capture the signal, but it's curious that everything works fine when not using Overmind.

The Dockerfile in question can be found here.

DarthSim commented 4 years ago

Hi,

It's not exactly Overmind bug. Overmind doesn't do anything fancy while interrupting processes, it just sends them SIGINT. When you hit ctrl +c, you send the signal directly to the process inside the container because your Stdin is attached to it. But when you send SIGINT to docker run process it just deattaches container and exits, don't know why. You can reproduce the same behavior without Overmind by sending SIGINT to docker run with kill -2.

There is a workaround for this case, but it looks a bit hacky. Just add this to your script:

trap 'docker stop websockify' EXIT > /dev/null

Or you can add the same to the Procfile:

websockify: trap 'docker stop websockify' EXIT > /dev/null; script/websockify-server