dumb-init and `go run`

[paymog]

I have an application that emits two log messages upon receiving a SIGINT or SIGTERM

{"Namespace":"default","Signal":"interrupt","TaskQueue":"main-task-queue","WorkerID":"37574@Paymahns-Air@","level":"error","msg":"Worker has been stopped.","time":"2021-06-17T12:19:27-05:00"}
{"Namespace":"default","TaskQueue":"main-task-queue","WorkerID":"37574@Paymahns-Air@","level":"error","msg":"Stopped Worker","time":"2021-06-17T12:19:27-05:00"}

the first message happens upon receiving the signal, the final message is emitted when the graceful shutdown has finished.

I find that when I run my application with dumb-init -- go run path/to/main.go and then send a SIGTERM/SIGINT only the first of these two log messages is emitted. I've also found that go run spawns a subprocess similar like so (note that in this example dumb-init wasn't used):

root@worker-ffzpxpdm-78b9797dcd-xsfwr:/gadic# ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.3  0.3 2046200 30828 ?       Ssl  18:04   0:12 go run src/cmd/worker/main.go --temporal-host temporal-server.temporal.svc.cluster.local --temporal-port 7233 --grpc-port 6789 --grpc-hos
root        3782  0.0  0.5 1640772 43232 ?       Sl   18:06   0:00 /tmp/go-build2661472711/b001/exe/main --temporal-host temporal-server.temporal.svc.cluster.local --temporal-port 7233 --grpc-port 6789 --
root        3808  0.1  0.0   4244  3468 pts/0    Ss   19:07   0:00 /bin/bash
root        3817  0.0  0.0   5900  2792 pts/0    R+   19:07   0:00 ps aux

Does dumb-init support signal forwarding and process waiting for processes that spawn other processes?

[chriskuehl]

By default it signals the entire process group it establishes in the child: https://github.com/Yelp/dumb-init#session-behavior

I don't know what go run does (e.g. if it establishes its own new process group in the child, which would defeat this) but you may want to check the process groups involved using ps.

It does not currently support waiting for all processes to exit other than the direct child (because it is difficult (impossible?) to implement with the available APIs in a race-free way). It's possible that go run just exits too fast when it gets those signals.

[paymog]

Looks like go run spawns a child process with the same group:

 ❮❮❮ ps aux -o user,pid,start,time,group,gid,command | rg test

paymahn          81865   0.0  0.0  4264552    288 s002  R+    3:47pm   0:00.00 rg -i test       paymahn          81865  3:47pm   0:00.00    20    20 rg -i test
paymahn          81290   0.0  0.1  5006524  11996 s007  S+    3:42pm   0:00.27 /var/folders/t9/ paymahn          81290  3:42pm   0:00.27    20    20 /var/folders/t9/s2v_yljn2xq8mm0nlh55k69r0000gn/T/go-build2423137104/b001/exe/test
paymahn          81259   0.0  0.2  5017900  17476 s007  S+    3:42pm   0:03.96 go run src/cmd/t paymahn          81259  3:42pm   0:03.96    20    20 go run src/cmd/test/test.go

I guess your theory of go run exiting too fast is probably correct.