search

BCDevOps / developer-experience

This repository is used to track all work for the BCGov Platform Services Team (This includes work for: 1. Platform Experience, 2. Developer Experience 3. Platform Operations/OCP 3)
Apache License 2.0
8 stars 17 forks source link

Nagios - Create notification for silenced alerts #4862

Open vivekratan88 opened 1 month ago

vivekratan88 commented 1 month ago

Describe the issue Check to see if there is way to create notification for alerts which are disabled manually and have not been turned ON

What is the Value/Impact? Will ensure we don't miss on any alerts that are silenced.

What is the plan? How will this get completed? Key components here is to check APIs used by Nagios to see if we can get this done

Identify any dependencies Incase of absence please contact Steven B,Will H or Tim B regarding the ticket.

Definition of done Complete should be email sent when notification is turned OFF for specific amount of time and host exists in Nagios

vivekratan88 commented 3 weeks ago

what is been silenced needs to be reported in Nagios also what is silenced also should be reported - david elash

vivekratan88 commented 2 weeks ago

Having a look at API docs to see if I can list all hosts whose notifications_enabled: can be set to zero, that will help in this case.

vivekratan88 commented 4 days ago

Got play written with RO API key, doing testing now on AAP some errors seen trying to fix it with Rowan and Steven's guidance.

vivekratan88 commented 3 days ago

More testing today, had to change API url from http to https and ran it the output seems to have been blank so working on adding FQDN to all modules and also provide debug output for host status response and re-run it again.