search

tiangolo / uwsgi-nginx-flask-docker

Docker image with uWSGI and Nginx for Flask applications in Python running in a single container.
https://hub.docker.com/r/tiangolo/uwsgi-nginx-flask/
Apache License 2.0
2.98k stars 608 forks source link

connect() to unix:///tmp/uwsgi.sock failed (111: Connection refused) while connecting to upstream, client: ----, server: , request: "POST /user HTTP/1.1", upstream: "uwsgi://unix:///tmp/uwsgi.sock:", host: "---" #192

Closed truckbook-llc closed 3 weeks ago

truckbook-llc commented 4 years ago

I am running this container which connects to database container. If the database container goes down for a few seconds or is not available initially when this container starts, all workers are killed and uwsgi starts giving error forever even if the database comes back up. Can you please help me identify how can I prevent this or reconnect to database automatically.

vagdevik commented 3 years ago

Same issue

vagdevik commented 3 years ago

In my case, I have defined the database host service as db in the flask app script, whereas the actual kubernetes service (that connects to the deployment of this app image) is named as postgres-db.

In my docker-compose, I have a service named db which is based on postgres image. This is linked to my flask web service in docker-compose. Then, I wanted to deploy the app on to minikube. I defined postgres-db service which should be set as env variable in the web deployment resource. Though did that, the image of the app I was using has the hard-coded code where the host is hardcoded as db. So I have set the env variables, fetched them in the code, re-built the image of app, and used the new image.

This is due to the overlooking this issue while building the image without using environment variables, whereas when working with the kubernetes I needed to declare these credentials as environment variables.

So just makesure that, the host name(or any other such data) is modified as per the environment variables defined in the deployment resources, and that the image(of the app) is built on top of the code which actually gets credential with the help of environment variable rather than getting them directly.

Correctly define the services, attach them to deployments, makesure the image is built on top of code using environment variables.

MiltiadisKoutsokeras commented 3 years ago

This error is present in my deployment in Google Cloud Run, and surfaces only on spin-up of a sleeping container. On startup, triggered by client HTTP request, the first response is always 502 until the service is up and all external services like database are ready. I cannot reproduce it locally, as my system seems to be faster on spin-up and the problem does not manifest. I will report back in case of a solution. I suspect this will be present in every Cloud provider that launches the container on traffic trigger.

MiltiadisKoutsokeras commented 3 years ago

The problem lies on the supervisor setup. The supervisor launches the uwsgi and nginx programs, but the NGINX may be ready to server prior to uWSGI, that may lead to request arriving on this time slice, to return 502 due to missing socket or upstream server instance. One fix is to check the running state and IPC socket prior to launching NGINX but it is not a silver bullet, as the Cloud client request may timeout in the meanwhile. Classic race condition issue.