search

lsst-uk / somerville-operations

User issue reporting and tracking for the Somerville Cloud
0 stars 0 forks source link

iLO Email alert activation #184

Open astrodb opened 2 months ago

astrodb commented 2 months ago

We need to setup iLO email alerts on all the machines in Somerville to notify us when machines have alerts. This requires setting up Postfix mail servers on the mgmt container and the admin node, so messages can be sent by the iLOs to be relayed through the admin node to the SMTP server on the mgmt container, which will then send the messages onwards.

astrodb commented 2 months ago

Postfix on mgmt container completed by Greg. Postfix on admin container completed by Mark.

GregBlow commented 1 month ago

Need to test email throughput from a single node, then configure all nodes to use same method (then determine how to push a test to all).

iLO configuration using an adaptation of https://git.ecdf.ed.ac.uk/gblow/ilo-license-setter should be useful.

astrodb commented 6 days ago

Just recalled an important thing to check when setting this up: be sure to set the 'SMTP reverse-path' in the SMTP settings on each machine to a mappable email address like 'root@somerville-mgmt.epcc.ed.ac.uk'. Otherwise the emails will be likely get bounced by the Uni relays for being malformed/non-existent (since the messages will be sent with some default 'admin@ip-addess' address in the header).