Handling Shutdown on Kubernetes

[saga-dash]

When running on Kubernetes, there is uncertainty regarding the order of processing between iptable changes and SIGTERM, which may lead to a situation where routing is closed immediately after receiving SIGTERM, causing new requests to receive a 503 error. We have confirmed that using preClose in fastify does not have any effect since all routing is already closed at that point.

I wanted to introduce a few seconds of wait time using fastify.gracefulShutdown, but it seems ineffective because the handlers using fastify.close are executed in parallel.

Alternatively, should I consider solving this issue by using the preStop container lifecycle hook in Kubernetes to introduce a sleep period?

Translated from Japanese

k8s上で動かす場合において、iptable変更とSIGTERMはどちらが先に処理されるか不明なため、SIGTERMが送られた直後にルーティングが閉じると新しいリクエストが503になる可能性がある。 fastifyのpreCloseはフックされた時点で全てのルーティングが閉じているため、preCloseに待機時間をいれても効果がないことは確認しています。 fastify.gracefulShutdownで数秒の待機時間を入れたいと思ったのだが、fastify.closeをしているハンドラーが並列で実行されているためうまくいかない。 それとも単純にk8sのcontainer lifecycle hooksのpreStopでsleepさせることで解決するべきでしょうか？

[StarpTech]

Hi @saga-dash, I don't understand how this library is related to iptable changes or kubernetes. Are you able to provide an isolated, reproducible example?

[saga-dash]

Thank you for your reply! Unfortunately, it seems that this issue is quite rare, and I haven't been able to come up with a reliable way to reproduce it. After giving it some thought overnight, I concluded that using the preStop sleep feature in Kubernetes would be the appropriate way to handle this problem.