Closed mdegat01 closed 3 years ago
So the problem with that is I don't have an easy way to figure out that port. When the user sets a value for the port then NGinx listens on 80
. But when the user does not and is running in Ingress mode then NGinx listens on 1337
. Neither of those values is in config so I can't use the normal placeholders of PORT
or a config value to get to the right spot.
To do that I think the easiest way would be to add a new server config for nginx that listens on a different unrelated port (81
? 8080
?). That config pass through requests which exactly matched /api/health
and reject everything else. Then I could use that URL for all configurations (Ingress and non-Ingress).
Let me know what you think about this approach.
The Ingress NGINX config is always enabled, if you assign a port it will configure both.
A user cannot disable Ingress.
Oh right, duh, its got .conf
not .disabled
. Ok cool, will use that, thanks!
Watchdog requests originate from supervisor not observer though right? Since observer runs at a different IP and so will be denied access at port 1337
Looks like its good. Updated it to use port 1337
and ran it locally for about a half an hour with watchdog on, didn't see any warnings.
Proposed Changes
Grafana has a /health API which does not require authentication and basically just returns whether Grafana is happily running or not. Setting the watchdog URL to this makes it so watchdog calls it regularly and restarts the addon if it receives an unhealthy response.
Since the internal port that grafana listens on needs to stay open I figured we might as well use it for these health checks since watchdog will always be able to reach grafana here, regardless of other configuration.
I pulled this branch locally to test and it seems to be running without issue. I guess the one very small note to make is that if someone has set the environmental variable
GF_SERVER_HTTP_ADDR
in their config then watchdog won't like it since it will be denied access on port3000
. That's actually how I confirmed it was working, I let addon run fine for about an hour then added this to the config and saw I got a warning from watchdog in my log:Related Issues