search

GameServerManagers / LinuxGSM

The command-line tool for quick, simple deployment and management of Linux dedicated game servers.
https://linuxgsm.com
MIT License
4.18k stars 800 forks source link

[Feature]: Rate Limit the number of alerts that can be sent #4566

Open MicLieg opened 2 months ago

MicLieg commented 2 months ago

User story

As a server admin, i don't want to get spammed with alerts by a server that isn't starting correctly.

Game

All

Linux distro

Ubuntu 22.04

Command

command: monitor

Further information

Currently, if a server repeatedly crashes or can't start properly, running the command `./gameserver monitor', LinuxGSM will send an alert that the server is offline if alerts are turned on. Whilst this is generally very helpful to be notified of problems, it can cause a massive flood of alerts if the monitor command is set to run every 5 minutes, as suggested in the Docs.

I suggest either implementing a hard rate limit for a certain period of time, so that only a configurable number of alerts can be sent within a configurable period of time, or temporarily disabling alerts if the monitor command fails a configurable number of times in succession.

irobot73 commented 1 month ago

Maybe a health check via Docker that we could then use the 'max_attempts' ala RESTART_POLICY (yeah, it's the server that's dorked...not the container)?

      # https://docs.docker.com/compose/compose-file/compose-file-v3/#restart_policy
      restart_policy:
        condition: on-failure
        delay: 10s #Default '5'
        max_attempts: 3