search

abahmed / kwatch

:eyes: monitor & detect crashes in your Kubernetes(K8s) cluster instantly
https://kwatch.dev
MIT License
944 stars 75 forks source link

False crash with reason "Unschedulable" #318

Closed divomen closed 3 weeks ago

divomen commented 3 weeks ago

Describe the bug When I increase number of replicas, kwatch send me notification in slack about crash with reason Unschedulable (befor autoscaler adds a node). I tried to disable this reason by config:

namespaces:
  - default
  - staging
  - demo
reasons:
  - "!Unschedulable"

But it still sends notification with this reason

To Reproduce Steps to reproduce the behavior: just disable this reason in config and increase number of replicas of a pod until there is not enough resources in current nodes

Expected behavior Not to send a notification with disabled reason

Actual behavior send a notification with disabled reason

Version/Commit 0.9.3

thomasLeclaire commented 3 weeks ago

Hi! Could be linked or not but I'm also experiencing some abnormal behavior when we use some autoscaling. with v 0.9.3 I get some alerts on any scale down even if pods are normally and expectedly stopped.

abahmed commented 3 weeks ago

Hello @thomasLeclaire Can you provide more details (e.g. logs of kwatch) in a separate issue?

abahmed commented 3 weeks ago

@divomen Should be fixed in Release v0.9.5

@thomasLeclaire Can you give a try with the new release? and if it still happening open an issue with details to be able to reproduce & fix it