Manechat / izzy-moonbot

Replacement admin bot for Manechat
https://manechat.net
MIT License
5 stars 5 forks source link

Docker Health Check Support #345

Open LunarNightShade opened 1 year ago

LunarNightShade commented 1 year ago

I'd like to be able to implement an automated Docker health check on Izzy's container for automated restarts should a fault state ever occur, such as encountering a gateway disconnect and not attempting to reconnect, like has occurred once before.

Docker needs a command that it can run against the container that will return a status code of 0 if the container's service is healthy. The example they give in the dockerfile reference is CMD curl -f http://localhost/ || exit 1 Basically, see if application is responding, else exit with status 1, telling Docker the service is dead and the container needs restarted.

The command used for the health check can be anything that can be executed in the container and return the correct exits codes (0 for success and 1 for unhealthy).

ASP.NET has a health check capability that exposes an http endpoint that Docker can curl inside the container to see if the application is alive or not. Relevant microsoft docs: https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks?view=aspnetcore-7.0 https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/monitor-app-health

From Discord messages:

https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks?view=aspnetcore-7.0#create-health-checks Izzy's connection disconnected, and never reconnected, but the app was still running.

Health checks are created by implementing the IHealthCheck interface. The CheckHealthAsync method returns a HealthCheckResult that indicates the health as Healthy, Degraded, or Unhealthy. The result is written as a plaintext response with a configurable status code. The health check's logic is placed in the CheckHealthAsync method. The preceding example sets a dummy variable, isHealthy, to true. If the value of isHealthy is set to false, the HealthCheckRegistration.FailureStatus status is returned.

izzy is already monitoring connection status to write connected / disconnected console messages write that status to a variable and have the health check logic aync method monitor it? if connected, return Healthy on the endpoint, if disconnected, return Degraded or Unhealthy on the endpoint Docker can check the endpoint once every say, 5 minutes (since a normal gateway reconnect event usually only takes a few seconds), and if it sees anything other than Healthy, it knows to kill and restart Izzy.

Ixrec commented 1 year ago

After trying and failing to follow the above instructions, my current impression is that none of this is available to us because we are not an ASP.NET app. If there's a way around that it's beyond me.