search

geerlingguy / raspberry-pi-dramble

DEPRECATED - Raspberry Pi Kubernetes cluster that runs HA/HP Drupal 8
http://www.pidramble.com/
MIT License
1.67k stars 260 forks source link

Add munin (or some other monitoring dashboard) #15

Closed geerlingguy closed 9 years ago

geerlingguy commented 9 years ago

It'd be nice to have a HUD of the entire cluster's activity. Munin is lightweight, easy to get going, and can maybe run on one of the three webservers, or on the caching server maybe...

geerlingguy commented 9 years ago

So... munin-html takes 100% CPU for about 20 seconds, then munin-graph takes over and spikes to 100% for about 3 minutes. A little longer, and the graphing cycle will basically be continuous!

Configuring Munin to use FastCGI with Nginx is not the most complicated affair (mostly involves maybe 4-6 more tasks in a playbook on the balancer... plus a bit more config in the Nginx .conf file...). This might be a deal-breaker, though, on the poor Pis.

I think we have four routes:

  1. Leave it, let one Pi core remain at basically 100% CPU almost forever. Doesn't have a huge impact on the balancer's ability to balance/cache requests, since CPU is not the bottleneck.
  2. Switch to CGI html/graphing strategy. This will make for slower individual graph loading, but the Pi will not be spiking in usage forever.
  3. See if I can disable the majority of 'nonessential' graphs that I don't care about (I care mostly about CPU, load, memory, disk IO, and network IO).
  4. Don't use munin to monitor the cluster's health. (Sad panda). Look for an alternative maybe.
geerlingguy commented 9 years ago

Another option worth exploring... Munin doesn't have a magic enabled var in munin.conf (sadly), but I could add a munin_enabled variable to the munin role, or a dramble_munin_enabled role specific to this configuration that removes the cron job. See: https://github.com/geerlingguy/ansible-role-munin/issues/2

geerlingguy commented 9 years ago

Done! Munin can now be toggled at-will by setting munin_cron_job to present or absent.