opencrvs / opencrvs-core

A global solution to civil registration
https://www.opencrvs.org
Other
79 stars 58 forks source link

Ensure mongo-on-update runs on each deployment #7256

Open euanmillar opened 2 days ago

euanmillar commented 2 days ago

Occasionally a server deployment can fail if mongo-on-update didnt run successfully, probably because mongo-on-update ran before mongo had fully initialised. This is hard to reproduce but sometimes we saw errors like this where Hearth could not connect to Mongo:

2023-02-23T13:24:57.598Z - error:  message=failed to connect to server [mongo1:27017] on first connect [MongoError: Authentication failed

The on-deploy.sh script creates users in the database and updates their passwords on each deploy.

A potential solution could be running the on-deploy.sh script after a 2 minute timeout to allow the stack a chance to fully start up. But it's still unclear what is causing this problem.