search

NixOS / infra

NixOS configurations for nixos.org and its servers
MIT License
230 stars 95 forks source link

eris: alert on failed systemd services #372

Closed delroth closed 7 months ago

delroth commented 7 months ago

There might be some initial flakiness due to some badly behaved services, e.g. hydra-scale-equinix-metal, but nothing that we can't silence away and properly fix later.

15m is a very conservative threshold, I'm expecting to push it down at some point once things get more under control.

delroth commented 7 months ago

Pushed for testing: https://monitoring.nixos.org/prometheus/alerts?search=systemd