search

fastapi / full-stack-fastapi-template

Full stack, modern web application template. Using FastAPI, React, SQLModel, PostgreSQL, Docker, GitHub Actions, automatic HTTPS and more.
MIT License
27.63k stars 4.92k forks source link

Docker Stack Deploy CI fails #145

Closed schorlton closed 4 years ago

schorlton commented 4 years ago

Using default settings and have Gitlab CI configured as per dockerswarm.rocks. This is awesome, and thank you for your hard work.

I'm running into an issue where Gitlab deploy job fails every time on the first try, but succeeds if I relaunch the deploy job. I think it may have something to do with dependency order? Here's my Gitlab CI log:

 Running with gitlab-runner 12.8.0 (1b659122)
   on Docker Runner 3itEe9vB
Using Docker executor with image tiangolo/docker-with-compose ...
00:01
 Pulling docker image tiangolo/docker-with-compose ...
 Using docker image sha256:7ebe8c2a97127972df9e803220f652a784238bb33ac1f6254dc18ddcc7c373b4 for tiangolo/docker-with-compose ...
Running on runner-3itEe9vB-project-16464367-concurrent-0 via e391f9defbe4...
00:01
Fetching changes with git depth set to 50...
00:03
 Reinitialized existing Git repository in /builds/samchorlton/bugseq-fastapi/.git/
 Checking out ffe949be as master...
 Removing docker-stack.yml
 Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
00:05
 WARNING! Using --password via the CLI is insecure. Use --password-stdin.
 WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
 Configure a credential helper to remove this warning. See
 https://docs.docker.com/engine/reference/commandline/login/#credentials-store
 Login Succeeded
 $ pip install docker-auto-labels
 Collecting docker-auto-labels
   Downloading https://files.pythonhosted.org/packages/8d/58/03eecdfa70fe31c1c9c46926bb5c70e6d450b07cead8a75065df1d959ad8/docker_auto_labels-0.2.3-py2.py3-none-any.whl
 Requirement already satisfied: pyyaml in /usr/lib/python2.7/site-packages (from docker-auto-labels) (3.13)
 Collecting Click>=6.0 (from docker-auto-labels)
   Downloading https://files.pythonhosted.org/packages/dd/c0/4d8f43a9b16e289f36478422031b8a63b54b6ac3b1ba605d602f10dd54d6/click-7.1.1-py2.py3-none-any.whl (82kB)
 Requirement already satisfied: docker in /usr/lib/python2.7/site-packages (from docker-auto-labels) (3.7.3)
 Requirement already satisfied: backports.ssl-match-hostname>=3.5; python_version < "3.5" in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (3.7.0.1)
 Requirement already satisfied: docker-pycreds>=0.4.0 in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (0.4.0)
 Requirement already satisfied: ipaddress>=1.0.16; python_version < "3.3" in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (1.0.23)
 Requirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (2.20.1)
 Requirement already satisfied: websocket-client>=0.32.0 in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (0.56.0)
 Requirement already satisfied: six>=1.4.0 in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (1.12.0)
 Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker->docker-auto-labels) (3.0.4)
 Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker->docker-auto-labels) (2019.9.11)
 Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker->docker-auto-labels) (1.24.3)
 Requirement already satisfied: idna<2.8,>=2.5 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker->docker-auto-labels) (2.7)
 Installing collected packages: Click, docker-auto-labels
 Successfully installed Click-7.1.1 docker-auto-labels-0.2.3
 $ DOMAIN=stag.bugseq.com TRAEFIK_TAG=stag.bugseq.com STACK_NAME=stag-bugseq-com TAG=stag sh ./scripts/deploy.sh
 Some services (backend, db, flower, frontend, pgadmin, proxy) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
 --- 
 === 
 Existing label: stag-bugseq-com.app-db-data 
 With existing value: true 
 In host: instance-1 
 With node ID: bnezvp5ptgwoein9trr7xvnjp 
 Matches required label: stag-bugseq-com.app-db-data 
 Declared with value: true 
 --- 
 Removing service stag-bugseq-com_backend
 Removing service stag-bugseq-com_celeryworker
 Removing service stag-bugseq-com_db
 Removing service stag-bugseq-com_flower
 Removing service stag-bugseq-com_frontend
 Removing service stag-bugseq-com_pgadmin
 Removing service stag-bugseq-com_proxy
 Removing service stag-bugseq-com_queue
 Removing network stag-bugseq-com_default
 Creating service stag-bugseq-com_proxy
 failed to create service stag-bugseq-com_proxy: Error response from daemon: network stag-bugseq-com_default not found
 ERROR: Job failed: exit code 1

Then hit retry:

 Running with gitlab-runner 12.8.0 (1b659122)
   on Docker Runner 3itEe9vB
Using Docker executor with image tiangolo/docker-with-compose ...
00:01
 Pulling docker image tiangolo/docker-with-compose ...
 Using docker image sha256:7ebe8c2a97127972df9e803220f652a784238bb33ac1f6254dc18ddcc7c373b4 for tiangolo/docker-with-compose ...
Running on runner-3itEe9vB-project-16464367-concurrent-0 via e391f9defbe4...
00:01
Fetching changes with git depth set to 50...
00:05
 Reinitialized existing Git repository in /builds/samchorlton/bugseq-fastapi/.git/
 Checking out 409f6f6e as master...
 Removing docker-stack.yml
 Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
00:13
 WARNING! Using --password via the CLI is insecure. Use --password-stdin.
 WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
 Configure a credential helper to remove this warning. See
 https://docs.docker.com/engine/reference/commandline/login/#credentials-store
 Login Succeeded
 $ pip install docker-auto-labels
 Collecting docker-auto-labels
   Downloading https://files.pythonhosted.org/packages/8d/58/03eecdfa70fe31c1c9c46926bb5c70e6d450b07cead8a75065df1d959ad8/docker_auto_labels-0.2.3-py2.py3-none-any.whl
 Requirement already satisfied: pyyaml in /usr/lib/python2.7/site-packages (from docker-auto-labels) (3.13)
 Collecting Click>=6.0 (from docker-auto-labels)
   Downloading https://files.pythonhosted.org/packages/dd/c0/4d8f43a9b16e289f36478422031b8a63b54b6ac3b1ba605d602f10dd54d6/click-7.1.1-py2.py3-none-any.whl (82kB)
 Requirement already satisfied: docker in /usr/lib/python2.7/site-packages (from docker-auto-labels) (3.7.3)
 Requirement already satisfied: backports.ssl-match-hostname>=3.5; python_version < "3.5" in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (3.7.0.1)
 Requirement already satisfied: docker-pycreds>=0.4.0 in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (0.4.0)
 Requirement already satisfied: ipaddress>=1.0.16; python_version < "3.3" in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (1.0.23)
 Requirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (2.20.1)
 Requirement already satisfied: websocket-client>=0.32.0 in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (0.56.0)
 Requirement already satisfied: six>=1.4.0 in /usr/lib/python2.7/site-packages (from docker->docker-auto-labels) (1.12.0)
 Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker->docker-auto-labels) (3.0.4)
 Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker->docker-auto-labels) (2019.9.11)
 Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker->docker-auto-labels) (1.24.3)
 Requirement already satisfied: idna<2.8,>=2.5 in /usr/lib/python2.7/site-packages (from requests!=2.18.0,>=2.14.2->docker->docker-auto-labels) (2.7)
 Installing collected packages: Click, docker-auto-labels
 Successfully installed Click-7.1.1 docker-auto-labels-0.2.3
 $ DOMAIN=stag.bugseq.com TRAEFIK_TAG=stag.bugseq.com STACK_NAME=stag-bugseq-com TAG=stag sh ./scripts/deploy.sh
 Some services (backend, db, flower, frontend, pgadmin, proxy) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
 --- 
 === 
 Existing label: stag-bugseq-com.app-db-data 
 With existing value: true 
 In host: instance-1 
 With node ID: bnezvp5ptgwoein9trr7xvnjp 
 Matches required label: stag-bugseq-com.app-db-data 
 Declared with value: true 
 --- 
 Nothing found in stack: stag-bugseq-com
 Creating network stag-bugseq-com_default
 Creating service stag-bugseq-com_flower
 Creating service stag-bugseq-com_frontend
 Creating service stag-bugseq-com_pgadmin
 Creating service stag-bugseq-com_proxy
 Creating service stag-bugseq-com_queue
 Creating service stag-bugseq-com_backend
 Creating service stag-bugseq-com_celeryworker
 Creating service stag-bugseq-com_db
 Job succeeded
tiangolo commented 4 years ago

It seems like it wasn't able to create the internal Docker Swarm network for the stack the first time.

First, you might want to check your Docker Compose files, recently I refactored them to use only 2 files that should simplify it all a bit and maybe help to find out what might be happening.

Then, you could post here your Docker Compose file so that we can see if there's anything else that could be interfering.

github-actions[bot] commented 4 years ago

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.