The server will not shut down until all jobs have finished running.
If the server does not shut down gracefully in some timeout k8s will kill 9 the pod. This will screw up any jobs which are running.
[ ] Increase the timeout to something pretty big, maybe 2 minutes, so that this never happens.
[ ] Look into k8s pods in the terminating state to make sure that no new connections are sent to these pods
If new connections are sent to terminating pods then simply setting the timeout to be higher will not be enough. As the server could receive a web hook request and start a new job, which in turn would make the server wait even longer before shutting off, one could imagine a situation where the terminating pod keeps receiving web hook requests and never gracefully shuts down, causing a kill 9
The server will not shut down until all jobs have finished running.
If the server does not shut down gracefully in some timeout k8s will kill
9the pod. This will screw up any jobs which are running.9