Enhancement: Instance metrics monitoring

[perennialtech]

Enhancement: Instance Metrics Monitoring

What would you like to be added?

Implement monitoring of instance metrics. Information such as uptime data and response times should be recorded and made available to users on this repository (redlib-org/redlib-instances).

Possible monitoring options include:

1. updown.io

• The most feature-rich solution out of the following and is easy to implement.
• Able to configure check frequency (between 15s and 1h) and response time checks from multiple locations, in addition to typical uptime monitoring.
• Exposes a unique metric called Apdex that measures performance on the client-side, which is useful for knowing if an instance is experiencing degraded performance.
• An account starts with 100,000 free requests, but then costs money for additional requests (relatively inexpensive, see the pricing page for more details).
• Status pages can be minimally customised.
• See example status page.

2. UptimeRobot

• A free and easy-to-set-up solution similar to updown.io, but with more restricted free features.
• The shortest monitoring interval is 5 minutes, and exposing long-term response time history appears to be impossible on the free plan (only data for the last 2 days are shown).
• Status pages can be minimally customised.
• See the official example status page or the deployment by the Invidious project for a real-world example that uses the free plan.

3. Upptime

• A free solution that uses GitHub Actions for monitoring, somewhat similar in features to updown.io.
• Checks every 5 minutes by default and records response times every 6 hours.
• Automatically opens GitHub Issues if an incident occurs.
  • The default is when an instance is unreachable (if the domain name fails to resolve, anything other than a 20x and 30x HTTP code is returned, etc.), but it can also be configured if response time exceeds a given value.
• Status pages can be greatly customised.
• See the official example status page or the deployment by the SearXNG project for a real-world example.

4. uptime-kuma

• A self-hosted solution that appears to be popular among instance maintainers.
• I haven't used this solution myself as I believe instance monitoring is a critical service that should not be self-hosted.

Other solutions may be considered, but I won't be able to attest to their usefulness.

As for my personal recommendations:

• If you're willing to put some money toward instance monitoring, use updown.io for the features it provides.

  • Long-term costs should be relatively low as there are only a few instances that require monitoring (checking 15 instances every 5 minutes will cost about 2.5€ per month once the free 100,000 requests are exhausted). Furthermore, these costs may be (partially) offset by donations to the Redlib project, if such a system is set up (currently, the links under the Sponsor this project section of redlib-org/redlib still point to those used by spikecodes).

  • A possible downside of going with updown.io would be the need to migrate to a different solution if the number of instances grows too large and the costs no longer outweigh the benefits (unlikely, as typically no more than 20 instances are hosting Redlib/Libreddit at a time).

• Otherwise, use UptimeRobot or Upptime as these solutions are easy to deploy and have a proven track record, as demonstrated by their long-term use in the Invidious and SearXNG projects, respectively. Choose UptimeRobot for a simple out-of-the-box solution, Upptime for one that is more feature-rich but requires additional configuration.

Note that a suitable endpoint should be chosen for monitoring (e.g., /r/popular); endpoints like /info are significantly cheaper for instances, but can miss instances that are online yet experiencing degraded real-world performance.

Why is this needed?

While information like geographical location is useful, instance metrics help people choose an instance based on reliability and performance using real-world data. Additionally, instances that fail to meet a reliability standard (e.g., >90% uptime over the last 30 days) may be removed from the list after a grace period to ensure service availability (this assumes Redlib itself is functional, but the instance is not; otherwise, this would make little sense).

[perennialtech]

Pinging @sigaloid for visibility regarding open issues on this repository.

[sigaloid]

Done: https://stats.uptimerobot.com/mpmqAs1G2Q