search

uselagoon / lagoon-charts

A collection of Helm charts for Lagoon and associated services.
Apache License 2.0
11 stars 10 forks source link

feat: apiDB/keycloakDB startup/readiness/liveness probes and startup/shutdown time #650

Closed rocketeerbkw closed 3 months ago

rocketeerbkw commented 3 months ago

Probes

There are some changes to all the probe types for both apiDB and keycloakDB to fix unnecessary warnings in logs and to prevent premature connections.

startupProbe

The init script from the uselagoon/mariadb base image starts mysqld multiple times to install/upgrade the schema, so a TCP probe will signal startup is complete before the init is actually done. The probe is changed to check for a file that signifies init completions. Depends on https://github.com/uselagoon/lagoon-images/pull/957.

livenessProbe

[Warning] Access denied for user 'root'@'127.0.0.1' (using password: YES)

Using localhost and port 3306 forces mysqladmin to connect to 127.0.0.1 via TCP, instead of via socket. The uselagoon/mariadb images only set a root password for localhost, so the attempt to authenticate via 127.0.0.1 fails.

readinessProbe

Aborted connection 178393 to db: 'unconnected' user: 'unauthenticated' host: '10.200.0.25' (This connection closed normally without authentication)

The current TCP based readiness probe causes an Aborted connection log for each check. Changing it to the readiness-probe.sh script removes the log and also ensures that mysqld is able to serve SQL requests, not just that it can open a socket.

Startup/Shutdown

In the case of a db with a lot of data that has a larger than default buffer pool and/or log file, there can be issues with the 30 second shutdown and startup timeouts that make it impossible to recover from a crash.

This PR makes the terminationGracePeriodSeconds configurable to allow more time for a complete shutdown.