search

mehdy / keepalived-exporter

Prometheus Keepalived exporter
GNU General Public License v3.0
121 stars 40 forks source link

Exporter not working when keepalived container init doesn't handle keepalived signals #72

Closed sbienkow closed 1 year ago

sbienkow commented 3 years ago

Currently, if docker mode is enabled, this exporter first checks signal numbers, by executing command in container: docker exec "${container-name}" keepalived --signum [STATS|DATA]. Related lines: https://github.com/cafebazaar/keepalived-exporter/blob/master/internal/types/container/keepalived_container_collector_host.go#L76-L104

It then uses those signal numbers to send them through docker to process with PID1 inside the container `docker kill : https://github.com/cafebazaar/keepalived-exporter/blob/master/internal/types/container/keepalived_container_collector_host.go#L106-L117

This unfortunately fails with osixia keepalived docker image, because it's init script doesn't relay these signals to keepalived instances it spawns https://github.com/osixia/docker-keepalived/issues/50. While I believe this should be fixed in the keepalived image itself, maybe it could be considered to be handled in the exporter. I've chosen to use that docker image, because it seems most up to date.

The fix for this seems relatively simple - instead of running docker kill --signal 12 keepalived, one could run docker exec keepalived sh -c 'kill -12 "$( cat /var/run/keepalived.pid )"'. Of course this assumes that sh exists, so would probably require it to be a separate command line argument to switch between these 2 behaviors.

mehdy commented 1 year ago

Sorry for the very long delay. We'll investigate soon.

@clwluvw Could you please investigate this issue?

clwluvw commented 1 year ago

We discussed that internally and agreed that this is the used keepalived image issue that is not running keepalived daemon as a pid 1 (which is the correct behavior in container runtime). So trying to support that bad practices would put us in a hole of dirty if/else which makes the project unmaintainable as an open source project (as it's going to accept source code changes from others).

As the mentioned image repo is deprecated (hasn't been updated for two years) you probably can build your own image (by replaying what the repo does or just simply installing keepalived from an alpine base which is not very outdated - e.g. https://github.com/mehdy/keepalived-exporter/blob/master/deployments/dev/keepalived.Dockerfile). Also forking the repo and adding the ability on your own is also an option.