Disk monitoring for EC2 instances

tcmaker / clubhouse

Prototype Member Dashboard

1 stars 1 forks source link

Disk monitoring for EC2 instances #46

Open automatingerror opened 2 years ago

automatingerror commented 2 years ago

In order to be aware of upcoming maintenance needs in case of log overflows or other issues that would fill up disk instances monitoring needs to be deployed

vandahm commented 2 years ago

Thinking about it, the solution doesn't need to be very complicated. We could just have a little script that runs as a cron job and emails the IT committee if disk usage rises above 90%.

techieb0y commented 2 years ago

Actions taken:

Installed Prometheus node exporter on the Clubhouse EC2 instance
Updated security group to allow traffic from the ELB to the instance on the exporter's port (9100)
Added rule to the ELB to permit the Comcast IP range to request /metrics on clubhouse.tcmaker.org and direct that to node_exporter instead of django
Added poller config to Prometheus on monitor.tcmaker.net

To do

Add alert manager rules to send notifications to Slack for disk space