sbrunner
commented
2 months ago The access logs should also easy be enabled: See: https://tomcat.apache.org/tomcat-9.0-doc/config/valve.html#Access_Logging
Open sbrunner opened 2 months ago
sbrunner
commented
2 months ago The access logs should also easy be enabled: See: https://tomcat.apache.org/tomcat-9.0-doc/config/valve.html#Access_Logging
Introduction
Currently, we get some lag around the observability of the application, then here we defined how it should be.
Notes that here we define the general framework, not all the specific cases event if we pout the first wanted implementations.
We target in priority the Kubernetes/Docker environment, then some words comes from this world.
Usage of the health checks.
This will update the Result or the
/metrics/healthcheckendpoint.Examples of the responses
When we call the
healthymethod:Response code: HTTP code
200.When we call the
unhealthymethod:Response code: HTTP status code
200(or500when we setJAVA_OPTSto-DhttpStatusIndicator=true)If we raise an exception:
Response code: HTTP code
200.Propose usage
Use this endpoint only in the Kubernetes endpoint to automatically restart the Pod.
Use the
heathyandunhealthymethod to change the view status.Add the option
httpStatusIndicator=truein the filecore/src/main/resources/mapfish-spring.properties.At first, we should get an error when we don't consume the queue event if she is not empty, at therm I think that this check should be something like this (for this check we need a time window.):
It's possible that we need a check that tests the building of an epsg code, in the past we get on some container this exception:
See also Jira issue.
Usage of the metric.
The metrics should be reviewer and documented, currently it's a little mess...
At first, we should add a gauge to observe the queue length and a timer to observe the total print duration.
Then we should review all the metrics, see if they're working, update/remove them if needed, add documentation.
Pertinent metric:
Current metrics:
HttpRequestFetcher:AbstractSingleImageLayer:CoverageTask:Cluster check
If we need a check, e.g. to notify that the print job queue it too long` we probably need to create a custom endpoint.
Resume
Use health checks only for health concerned with the current container.
Identify and add missing metrics to be able to better monitor the application with tools like Prometheus/Grafana.
Eventually, add a new endpoint for more specific checks like too long queue.