search

elastisys / compliantkubernetes-apps

Elastisys Compliant Kubernetes is an open source, Certified Kubernetes distribution designed according to the ISO27001 controls: providing you with security tooling and observability from day one.
https://elastisys.io/compliantkubernetes/
Apache License 2.0
46 stars 7 forks source link

Allow application developers to restart alertmanager #1935

Open Xartos opened 10 months ago

Xartos commented 10 months ago

Current behaviour

Currently when an application developer updates the alertmanager configuration secret there's no mechanism that reloads the config. And they are not allowed to restart the pod to apply that configuration.

Expected behaviour

An application developer can update the user-alertmanager config and the config should be reloaded. Either via some automatic config reload-sidecar or to allow restart the pod to apply that config.

Steps to reproduce

  1. As an application developer
  2. Update alertmanager config
  3. Try to restart the pod to apply the config
  4. Fail

Compliant Kubernetes Apps Version

v0.33.1

Kubernetes Version

v1.27.5

Additional context

No response

simonklb commented 10 months ago

Worth considering some mechanism that automatically reloads user-alertmanager when the config has changed?

Xartos commented 10 months ago

Worth considering some mechanism that automatically reloads user-alertmanager when the config has changed?

I'll update the issue to have the one doing this testing if this is possible first

OlleLarsson commented 10 months ago

Wait what, alertmanager at least has a container that is called config-reloader. And since this hasn't been an issue in the past it should work just fine unless something has been broken in a recent release.

Xartos commented 10 months ago

Wait what, alertmanager at least has a container that is called config-reloader. And since this hasn't been an issue in the past it should work just fine unless something has been broken in a recent release.

Hmm :thinking: I guess this will become a investigation task that it still works then. The alertmanager I saw this for was also broken in another way so it might have been that

OlleLarsson commented 10 months ago

Wait what, alertmanager at least has a container that is called config-reloader. And since this hasn't been an issue in the past it should work just fine unless something has been broken in a recent release.

Hmm 🤔 I guess this will become a investigation task that it still works then. The alertmanager I saw this for was also broken in another way so it might have been that

Let's hope that it was just that particular alertmanager that was broken 🤞🏻 If it's a wide-spread issue it feels like we should have seen more people complain about it.