openfun / richie

:pencil: An opensource CMS to build education portals
https://richie.education
MIT License
256 stars 80 forks source link

🔨(docker) fix first bootstrap can't connect to DB #2456

Open igobranco opened 3 days ago

igobranco commented 3 days ago

Fix error dockerflow.health.E001 with the message of Could not connect to database. This only occurs from a fresh installation or on the first boot of the database. Probably this only occurs when the dev machine has a slower IO. Split docker compose file, per supported database and add a dependency of the app to db, if it's healthy, directly on docker compose level.

fixes #2455

igobranco commented 3 days ago

With this PR it's possible to remove the use of @$(WAIT_DB) on the Makefile.

We could also add a healthcheck on docker-compose.yml file for the elasticsearch container and remove the @$(WAIT_ES) on the Makefile.

lunika commented 1 day ago

Thanks for your contribution. If we still use dockerize we can keep everything in one docker compose file. IMO it's easier to maintain in a single place instead of having to do it in multiple files. Keeping the healthcheck on both services can be a good idea. Having the best of the two modifications you made :

WDYT ?

igobranco commented 1 day ago

@lunika

I understand, but the dockerize is the reason of this problem. Next, is the explanation: The dockerize just runs a check port, this means that the service has started, but it doesn't check if that service is ready to receive traffic. This is more relevant on the first startup of a database, specially on a dev system with slow IO. When it is replaced with docker healthcheck command, each command check if that service is ready. So I think replacing the dockerize with healthcheck is the way to go, but this is in my opinion.

About the split of docker compose files: I tried to still have a single docker-compose.yml with everything, but it isn't possible to have a depends_on with long syntax with an if (postgresql or mysql). We need the long syntax so the app depends on a healthy database, so the only option to have multiple databases was to split the docker compose file. The env var can't be interpolated on docker compose yaml keys. Ref: https://docs.docker.com/compose/compose-file/05-services/#depends_on

I added the health check for elasticsearch, app, redis and redis sentinel services.

The dockerize can be replaced with docker compose up -d <service> --wait. So i went ahead and replaced it.

Now, with all database services with a proper healthcheck, and using docker wait it is more robust. Se aren't going to start anything with any dependent service not ok.