rfcx / rfcx-api

Core, Media/Assets and MQTT APIs
https://api.rfcx.org/docs/
Apache License 2.0
0 stars 0 forks source link

feat: graceful shutdown #608

Closed veckatimest closed 4 months ago

veckatimest commented 4 months ago

āœ… DoD

(use na when API docs (Release notes, etc) do not need to be updated)

šŸ“ Summary

According to this blog post (and some others) https://blog.risingstack.com/graceful-shutdown-node-js-kubernetes/ We should handle SIGTERM to finish handling express requests, close server and then shutdown. Even if the pod will not shutdown instantly, it'll change it's readiness probe status, which will stop incoming requests. So this should help us finish current activities. https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-best-practices-terminating-with-grace This article says that once pod is marked for deletion, it's excluded from Services. https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/ and this article says that default value for terminationGracePeriodSeconds is 30 seconds.

So this solution

  1. might help server to shutdown gracefully (by utilizing SIGTERM event handler)
  2. might help to understand whether or not the pods are killed by horizontal scaler (or is it called scheduler?)

šŸ“ø Examples

Put screenshots or response/request examples here!

šŸ›‘ Problems

šŸ’” More ideas

Write any more ideas you have