thaum-xyz / ankhmorpork

@paulfantom's GitOps managed kube cluster running in a cupboard. Built with fancy tools :sparkles:

https://ankhmorpork.thaum.xyz

MIT License

76 stars 10 forks source link

Alert: KubePodNotReady in monitoring #201

Open paulfantom opened 1 year ago

paulfantom commented 1 year ago

Alert KubePodNotReady firing in monitoring namespace

This is an automated issue created by the monitoring system. Please do not edit this message.

Alertmanager URL: https://alertmanager.ankhmorpork.thaum.xyz

Issue was last updated at 2024-09-02 15:05:18.111067281 +0000 UTC m=+5283.918553123.

Common Labels

alertname	KubePodNotReady
cluster	ankhmorpork
namespace	monitoring
pod	prometheus-k8s-1
prometheus	monitoring/k8s
severity	warning

Common Annotations

description	Pod monitoring/prometheus-k8s-1 has been in a non-ready state for longer than 15 minutes.
runbook_url	https://runbooks.thaum.xyz/runbooks/kubernetes/kubepodnotready
summary	Pod has been in a non-ready state for more than 15 minutes.

Alerts

StartsAt	Links
2024-08-26 08:34:39.821 +0000 UTC	GeneratorURL

paulfantom commented 1 year ago

Caused by overloaded node during cluster reboot. Fixed by cordoning node, removing offending pods, and uncordoning node to allow pod to start properly.

Should be prevented by implementing proper memory limits after cluster resize.