search

Smithx10 / nomad-driver-triton

HashiCorp Nomad Triton driver plugin
15 stars 6 forks source link

Handle Nomad Lifecycle Signals #9

Open Smithx10 opened 5 years ago

Smithx10 commented 5 years ago

I noticed an allocation's Event history, that Restart Signaled is firing off unix signals to try and bounce the task by sending unix signals. I need to interpret these and convert them into the proper restarting of the instance. This will require changing the exit behaviour currently, since if the triton Instance State ever goes to stopped Nomad will believe the allocation has exited. We could handle this by requiring nomad to read the instance state of stopped 3 times for example.

Recent Events: Time Type Description 2019-05-07T14:46:31-04:00 Started Task started by client 2019-05-07T14:45:40-04:00 Restarting Task restarting in 16.462999931s 2019-05-07T14:45:29-04:00 Terminated Exit Code: 0 2019-05-07T14:44:51-04:00 Restart Signaled healthcheck: check "service: \"consul-ssh\" check" unhealthy 2019-05-07T14:42:59-04:00 Started Task started by client 2019-05-07T14:42:16-04:00 Task Setup Building Task Directory 2019-05-07T14:42:16-04:00 Received Task received by client

Docs on Signals. https://www.nomadproject.io/guides/operating-a-job/update-strategies/handling-signals.html

Smithx10 commented 5 years ago

While investigating nomad some more, it looks like the signals it supports are just for the termination process and don't actually send SIGHUP or anything fo that nature. For the time being.... I'm closing this since this is not required.