search

NationalMuseumAustralia / Collection-API

The public web API of the National Museum of Australia
10 stars 0 forks source link

Monitor API and notify promptly when it goes down #124

Closed Conal-Tuohy closed 5 years ago

Conal-Tuohy commented 5 years ago

Note that relying on Nagios daemon on the host itself is not the best, since a server outage would take it down as well, and a network partition cutting it off from the internet would not necessarily be noticed, or allow a notification to be sent. Something on external infrastructure would be better.

Conal-Tuohy commented 5 years ago

Added script for running on oceania.digital.

https://github.com/NationalMuseumAustralia/Collection-API/blob/192eb5306fe412ea3872a8b079c510b1a042b502/monitoring/check-api.sh#L9

Still to do, update API shim to add a status endpoint separate from the other entities in the API, so we can replace the current monitor URL (object?text=*) with a request to status, which can report on Solr's availability without also writing to the API usage log.

Conal-Tuohy commented 5 years ago

Done. Script is running on oceania.org on a 5 minute repeating schedule, monitoring both test and production instances. Notification emails are sent to me and to the NMA API mailbox.

Conal-Tuohy commented 5 years ago

This is tested and complete