redpanda-data / kminion

KMinion is a feature-rich Prometheus exporter for Apache Kafka written in Go. It is lightweight and highly configurable so that it will meet your requirements.
MIT License
610 stars 122 forks source link

Add graceful shutdown to HTTP server #178

Closed hhromic closed 1 year ago

hhromic commented 1 year ago

While troubleshooting #177, I noticed that graceful shutdown was missing for the HTTP server, making KMinion to hang on interrupt signals, e.g. Ctrl-c, instead of shutting down properly. For example:

# When using Ctrl-c _before_ the HTTP server is started, KMinion shutdowns cleanly
(...)
{"level":"info","ts":"2022-12-13T10:00:49.245Z","logger":"main","msg":"successfully connected to kafka cluster"}
<Ctrl-c>
{"level":"info","ts":"2022-12-13T10:00:49.940Z","logger":"main","msg":"received a signal, going to shut down KMinion"}
{"level":"fatal","ts":"2022-12-13T10:00:50.179Z","logger":"main","msg":"failed to start minion service","error":"failed to check feature compatibility against Kafka: kafka api versions couldn't be fetched: failed to request api versions: context canceled"}

# KMinion exits and returns control to the console
# When using Ctrl-c _after_ the HTTP server is started, KMinion never shutdowns and just hangs
(...)
{"level":"info","ts":"2022-12-13T10:01:13.735Z","logger":"main","msg":"listening on address","listen_address":":8080"}
<Ctrl-c>
{"level":"info","ts":"2022-12-13T10:01:14.410Z","logger":"main","msg":"received a signal, going to shut down KMinion"}

# KMinion never exits from this point and control never returns to the console

This PR fixes the above by properly implementing HTTP server shutdown when the context is cancelled.

CLAassistant commented 1 year ago

CLA assistant check
All committers have signed the CLA.