samber / awesome-prometheus-alerts

🚨 Collection of Prometheus alerting rules
https://samber.github.io/awesome-prometheus-alerts/
Other
6.44k stars 987 forks source link

ContainerVolumeUsage always alarms #326

Open astiphout opened 1 year ago

astiphout commented 1 year ago

Hi, trying to understand the rule for ContainerVolumeUsage:

This always is zero, which causes the calculation to always be 100 sum by (instance) (container_fs_inodes_free{name!=""}) = 0 sum by (instance) (container_fs_inodes_total) = 95469464 1 - (sum by (instance) (container_fs_inodes_free{name!=""}) / sum by (instance) (container_fs_inodes_total)) = 1 (1 - (sum by (instance) (container_fs_inodes_free{name!=""}) / sum by (instance) (container_fs_inodes_total))) 100 > 80 = 100 (1 - 0 / 95469464) 100 = 100

At least in my environment (docker) it seems not working due to reported container_fs_inodes_free to be 0 for every container. I suspect the alarm is fine but something with cadvisor or working as designed? Looking for feedback, disabling for now

baurmatt commented 4 months ago

Having the same problem. cadvisor is deployed like this:

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:v0.49.1
    container_name: cadvisor
    restart: unless-stopped
    privileged: true
    pid: host
    networks:
      - backend
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
    devices:
      - /dev/kmsg