search

yyyar / gobetween

:cloud: Modern & minimalistic load balancer for the Сloud era
http://gobetween.io
Other
1.91k stars 210 forks source link

K8s shutdown #294

Open SVyatoslavG opened 3 years ago

SVyatoslavG commented 3 years ago

Added environment variable to support k8s graceful shutdown. It is not a graceful shutdown by definition. It would be useful for running gobetween behind Google Cloud Loadbalancer. Kubernetes Engine sends syscall.SIGTERM and at the same time stop sending requests to the pod before shutdown. Because those actions are done at the same time race condition occurs and the process needs to sleep before exiting. Please accept simple pull request.

yyyar commented 3 years ago

Kubernetes Engine sends syscall.SIGTERM and at the same time stop sending requests to the pod before shutdown. Because those actions are done at the same time race condition occurs and the process needs to sleep before exiting.

Could you please clarify why the process needs to sleep before exiting? So that we make sure gobetween won't shutdown while Kubernetes still sending requests to it? And also what value should GOBETWEEN_SHUTDOWN_TIME have - are there some recommendations for those who use Google Cloud Loadbalancer? Thanks!

SVyatoslavG commented 3 years ago

Sorry for the delay in answer. Yes, Kubernetes still sending requests to pods that are about to get deleting. SIGTERM & routing of requests happen at the same time and cause race conditions. I am using GOBETWEEN_SHUTDOWN_TIME 20s in my production environment. It is extremely hight for safety (theoretically GKE doesn't route any requests after 1s but for safety, I set 20s in case there are long-running requests).