sown / tasks

Tasks for sown projects
0 stars 0 forks source link

Monitor RAID arrays #57

Closed TimStallard closed 1 year ago

TimStallard commented 4 years ago

vms, vms-b53-1 and auth2 all use hardware raid 1 - we should be monitoring the health of these arrays.

Looks like nagios-plugins-contrib contains checks we could use, though relying on utilities for individual controllers.

TimStallard commented 4 years ago

Seems easy enough to get going:

root@vms-b53-1:/opt/raid# wget https://docs.broadcom.com/docs-and-downloads/raid-controllers/raid-controllers-common-files/8-07-14_MegaCLI.zip
root@vms-b53-1:/opt/raid# unzip 8-07-14_MegaCLI.zip 
root@vms-b53-1:/opt/raid# cd Linux/
root@vms-b53-1:/opt/raid/Linux# rpm2cpio MegaCli-8.07.14-1.noarch.rpm | cpio -idmv
root@vms-b53-1:/opt/raid/Linux# mv opt/MegaRAID /opt/

Then for the nagios side, install check_raid, generate sudo config and test:

root@vms-b53-1:/opt# apt install --no-install-recommends nagios-plugins-contrib
root@vms-b53-1:/opt# /usr/lib/nagios/plugins/check_raid -S -p megacli
Updating file /etc/sudoers.d/check_raid
/etc/sudoers.d/check_raid.new.18809: parsed OK
/etc/sudoers.d/check_raid file updated.
check_raid OK - sudoers updated

root@vms-b53-1:/opt# sudo -u nagios /usr/lib/nagios/plugins/check_raid -p megacli
OK: megacli:[Volumes(1): DISK0.0:Optimal,WriteCache:DISABLED; Devices(2): 00,01=Online]
root@vms-b53-1:/opt# echo $?
0

Will need some thought on how to handle in netbox and automate the checks on monitor2 (or just make them manual?)

TimStallard commented 4 years ago

All done on VMS, ignore on AUTH2 for now

TimStallard commented 1 year ago

the new routers also have hardware RAID, so I think this would be a good opportunity to do this properly and get the setup in ansible