Complete deployment instructions for DockerSwarm/Traefik/HTTPS on AWS EC2

[abrichr]

We had some issues getting this project deployed into production with HTTPS, and judging from the number of related issues, it seems we are not the only ones. Here are step-by-step instructions we used to deploy on a fresh Ubuntu 20.04 AWS EC2 instance:

sudo su -

# Configuration
export BASE_DOMAIN=foo.com
export BASE_NAME=foo-com
export ENV_NAME_SHORT=stag
export ENV_NAME_LONG=staging
export TRAEFIK_USERNAME=admin
export TRAEFIK_PASSWORD=<password>
export TRAEFIK_EMAIL=admin@$BASE_DOMAIN

# Clone your repo
# Assuming it is hosted at gitlab.com:
ssh-keygen -t rsa -b 2048 -C "me@foo.com"
cat ~/.ssh/id_rsa.pub
# Copy and paste the output of the above into a new key at https://gitlab.com/-/profile/keys
git clone git@gitlab.com:foo/bar.git
cd bar
nano .env
# paste env vars

# Configure your DNS, e.g.:
# CNAME stag ec2-...amazonaws.com.
# CNAME traefik stag.foo.com

# You should not need to modify anything below this line

# From https://dockerswarm.rocks/
export USE_HOSTNAME=$ENV_NAME_SHORT.$BASE_DOMAIN
echo $USE_HOSTNAME > /etc/hostname
hostname -F /etc/hostname
apt-get update
apt-get upgrade -y
curl -fsSL get.docker.com -o get-docker.sh
CHANNEL=stable sh get-docker.sh
rm get-docker.sh
docker swarm init
docker node ls

# From https://dockerswarm.rocks/traefik/
docker network create --driver=overlay traefik-public
export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add traefik-public.traefik-public-certificates=true $NODE_ID
export EMAIL=$TRAEFIK_EMAIL
export DOMAIN=traefik.$ENV_NAME_SHORT.$BASE_DOMAIN
export USERNAME=$TRAEFIK_USERNAME
export PASSWORD=$TRAEFIK_PASSWORD
export HASHED_PASSWORD=$(openssl passwd -apr1 $PASSWORD)
curl -L dockerswarm.rocks/traefik.yml -o traefik.yml
docker stack deploy -c traefik.yml traefik
docker stack ps traefik

# From https://docs.docker.com/compose/install/
curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
source ~/.profile
docker-compose --version

# The deploy script requires this library
apt install -y python3-pip
pip3 install docker-auto-labels

# Deploy your project
export TAG=$ENV_NAME_SHORT
# this isn't mentioned anywhere in the docs, and the script won't complain because DOMAIN was already set to traefik.foo.com
export DOMAIN=$ENV_NAME_SHORT.$BASE_DOMAIN
export FRONTEND_ENV=$ENV_NAME_LONG
bash scripts/build.sh
export TRAEFIK_TAG=$ENV_NAME_SHORT.$BASE_DOMAIN
export STACK_NAME=$ENV_NAME_SHORT-$BASE_NAME
bash scripts/deploy.sh

# If you get an error after the previous line, you may need to modify
# docker-compose.yml:
#   - change `version` to 3.6
#   - remove `depends_on` sections
# Then re-run the previous line

# Tail logs
docker service logs traefik_traefik -f

You should now be able to see your app at https://stag.foo.com, and the Traefik UI at https://traefik.stag.foo.com. It may take a couple of minutes for the certificate to become valid.

When changes are pushed to your repo, the following seems to be required in order to deploy:

git stash
git pull
git stash pop
bash scripts/build.sh
bash scripts/deploy.sh
docker service update --force $ENV_NAME_SHORT-${BASE_NAME}_backend -d
docker service update --force $ENV_NAME_SHORT-${BASE_NAME}_frontend -d

For completeness, here are the Docker versions:

$ docker --version
Docker version 19.03.13, build 4484c46d9d

$ docker-compose --version
docker-compose version 1.27.4, build 40524192

Hope someone finds this helpful!

[jceyrac]

Thanks a lot for this example @abrichr ! But unfortunately I think I'm missing something in the configuration. Let me try the explain. I'm trying to setup a staging environment accessible at the domain https://stag.verva.fr but I can't manage to connect to the app over https: Same for api endpoints or the documentation stag.verva.fr/docs:

Even though I can access the Traefik UI over https:

# Configure your DNS:
# A @ _<IP of the swarm node>_
# CNAME stag node....ruk-com.cloud
# CNAME www node....ruk-com.cloud
# CNAME traefik stag.verva.fr

# From https://dockerswarm.rocks/traefik/
docker network create --driver=overlay traefik-public
export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add traefik-public.traefik-public-certificates=true $NODE_ID
export EMAIL=jceyrac@protonmail.com
export DOMAIN=traefik.verva.com
export USERNAME=admin
export PASSWORD=_<password>_
export HASHED_PASSWORD=$(openssl passwd -apr1 $PASSWORD)
curl -L dockerswarm.rocks/traefik.yml -o traefik.yml
docker stack deploy -c traefik.yml traefik
docker stack ps traefik

# Deploy your project
export TAG=stag
# this isn't mentioned anywhere in the docs, and the script won't complain because DOMAIN was already set to traefik.foo.com
export DOMAIN=stag.verva.fr
export FRONTEND_ENV=staging
bash scripts/build.sh
export TRAEFIK_TAG=stag.verva.fr
export STACK_NAME=stag-verva-fr
vim docker-compose.yml
# change version to 3.6

What do you mean by "# remove depends_on"? I haven't removed anything in the docker-compose.yml.

Do you see anything wrong?

[abrichr]

If you don't get an error about depends_on when deploying then don't worry about changing docker-compose.yml.

It looks like the site is being served over https, but the certificate is invalid. It may take a few minutes for the certificate provided by LetsEncrypt to become valid, and if you accessed it before then, the certificate might be cached. Try accessing it from a different browser/computer. If it still isn't working take a look at the Traefik logs with docker service logs traefik_traefik.

There might also be more information in Firefox. Try clicking on the arrow to the right of the message in the first screenshot and see what it says.

(I would look myself but it looks like https://stag.verva.fr/ is currently down.)

Edit: looking at your commands above, it seems that this line:

export DOMAIN=traefik.verva.com

Should be:

export DOMAIN=traefik.stag.verva.fr

I've updated my original comment so that you only need to specify each bit of information once.

[jceyrac]

Thanks for your reply @abrichr and the update of the commands. Just before you came back to me I gave up setting up the staging environment to try to setup the the environment on the production one, on the domain verva.fr.

I think I'm close to make it work since I now seem to be able to call all the following over HTTPS:

• https://verva.fr
• https://traefik.verva.fr
• https://verva.fr/docs , calls to the fastapi services with Postman over https
• https://strapi.verva.fr/coins , calls to the strapi services for cms

But I still have an issue with the frontend which is throwing the error "Blocked loading mixed active content “http://verva.fr/api/v1/prices/”: .

My understanding of this error is that the frontend should be calling this services of the fastapi over HTTPS but it is calling it over HTTP. is it correct?

Following the comment of @wolfieorama https://github.com/tiangolo/full-stack-fastapi-postgresql/issues/239#issuecomment-694032115 I updated my docker-compose.yml with traefik.docker.network and traefik.docker.router labels but without any success:

version: "3.6"
services:

  proxy:
    image: traefik:v2.2
    networks:
      - ${TRAEFIK_PUBLIC_NETWORK?Variable not set}
      - default
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command:
      # Enable Docker in Traefik, so that it reads labels from Docker services
      - --providers.docker
      # Add a constraint to only use services with the label for this stack
      # from the env var TRAEFIK_TAG
      - --providers.docker.constraints=Label(`traefik.constraint-label-stack`, `${TRAEFIK_TAG?Variable not set}`)
      # Do not expose all Docker services, only the ones explicitly exposed
      - --providers.docker.exposedbydefault=false
      # Enable Docker Swarm mode
      - --providers.docker.swarmmode
      # Enable the access log, with HTTP requests
      - --accesslog
      # Enable the Traefik log, for configurations and errors
      - --log
      # Enable the Dashboard and API
      - --api
    deploy:
      placement:
        constraints:
          - node.role == manager
      labels:
        # Enable Traefik for this service, to make it available in the public network
        - traefik.enable=true
        # Use the traefik-public network (declared below)
        - traefik.docker.network=${TRAEFIK_PUBLIC_NETWORK?Variable not set}
        # Use the custom label "traefik.constraint-label=traefik-public"
        # This public Traefik will only use services with this label
        - traefik.constraint-label=${TRAEFIK_PUBLIC_TAG?Variable not set}
        # traefik-http set up only to use the middleware to redirect to https
        - traefik.http.middlewares.${STACK_NAME?Variable not set}-https-redirect.redirectscheme.scheme=https
        - traefik.http.middlewares.${STACK_NAME?Variable not set}-https-redirect.redirectscheme.permanent=true
        # Handle host with and without "www" to redirect to only one of them
        # Uses environment variable DOMAIN
        # To disable www redirection remove the Host() you want to discard, here and
        # below for HTTPS
        - traefik.http.routers.${STACK_NAME?Variable not set}-proxy-http.rule=Host(`${DOMAIN?Variable not set}`) || Host(`www.${DOMAIN?Variable not set}`)
        - traefik.http.routers.${STACK_NAME?Variable not set}-proxy-http.entrypoints=http
        # traefik-https the actual router using HTTPS
        - traefik.http.routers.${STACK_NAME?Variable not set}-proxy-https.rule=Host(`${DOMAIN?Variable not set}`) || Host(`www.${DOMAIN?Variable not set}`)
        - traefik.http.routers.${STACK_NAME?Variable not set}-proxy-https.entrypoints=https
        - traefik.http.routers.${STACK_NAME?Variable not set}-proxy-https.tls=true
        # Use the "le" (Let's Encrypt) resolver created below
        - traefik.http.routers.${STACK_NAME?Variable not set}-proxy-https.tls.certresolver=le
        # Define the port inside of the Docker service to use
        - traefik.http.services.${STACK_NAME?Variable not set}-proxy.loadbalancer.server.port=80
        # Handle domain with and without "www" to redirect to only one
        # To disable www redirection remove the next line
        - traefik.http.middlewares.${STACK_NAME?Variable not set}-www-redirect.redirectregex.regex=^https?://(www.)?(${DOMAIN?Variable not set})/(.*)
        # Redirect a domain with www to non-www
        # To disable it remove the next line
        - traefik.http.middlewares.${STACK_NAME?Variable not set}-www-redirect.redirectregex.replacement=https://${DOMAIN?Variable not set}/$${3}
        # Redirect a domain without www to www
        # To enable it remove the previous line and uncomment the next
        # - traefik.http.middlewares.${STACK_NAME}-www-redirect.redirectregex.replacement=https://www.${DOMAIN}/$${3}
        # Middleware to redirect www, to disable it remove the next line 
        - traefik.http.routers.${STACK_NAME?Variable not set}-proxy-https.middlewares=${STACK_NAME?Variable not set}-www-redirect
        # Middleware to redirect www, and redirect HTTP to HTTPS
        # to disable www redirection remove the section: ${STACK_NAME?Variable not set}-www-redirect,
        - traefik.http.routers.${STACK_NAME?Variable not set}-proxy-http.middlewares=${STACK_NAME?Variable not set}-www-redirect,${STACK_NAME?Variable not set}-https-redirect

  db:
    image: postgres:12
    volumes:
      - app-db-data:/var/lib/postgresql/data/pgdata
    env_file:
      - .env
    environment:
      - PGDATA=/var/lib/postgresql/data/pgdata
    deploy:
      placement:
        constraints:
          - node.labels.${STACK_NAME?Variable not set}.app-db-data == true

  pgadmin:
    image: dpage/pgadmin4
    networks:
      - ${TRAEFIK_PUBLIC_NETWORK?Variable not set}
      - default
    #depends_on:
    #  - db
    env_file:
      - .env
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${TRAEFIK_PUBLIC_NETWORK?Variable not set}
        - traefik.constraint-label=${TRAEFIK_PUBLIC_TAG?Variable not set}
        - traefik.http.routers.${STACK_NAME?Variable not set}-pgadmin-http.rule=Host(`pgadmin.${DOMAIN?Variable not set}`)
        - traefik.http.routers.${STACK_NAME?Variable not set}-pgadmin-http.entrypoints=http
        - traefik.http.routers.${STACK_NAME?Variable not set}-pgadmin-http.middlewares=${STACK_NAME?Variable not set}-https-redirect
        - traefik.http.routers.${STACK_NAME?Variable not set}-pgadmin-https.rule=Host(`pgadmin.${DOMAIN?Variable not set}`)
        - traefik.http.routers.${STACK_NAME?Variable not set}-pgadmin-https.entrypoints=https
        - traefik.http.routers.${STACK_NAME?Variable not set}-pgadmin-https.tls=true
        - traefik.http.routers.${STACK_NAME?Variable not set}-pgadmin-https.tls.certresolver=le
        - traefik.http.services.${STACK_NAME?Variable not set}-pgadmin.loadbalancer.server.port=5050

  queue:
    image: rabbitmq:3
    # Using the below image instead is required to enable the "Broker" tab in the flower UI:
    # image: rabbitmq:3-management
    #
    # You also have to change the flower command

  flower:
    image: mher/flower:0.9.4
    networks:
      - ${TRAEFIK_PUBLIC_NETWORK?Variable not set}
      - default
    env_file:
      - .env
    command:
      - "--broker=amqp://guest@queue:5672//"
      # For the "Broker" tab to work in the flower UI, uncomment the following command argument,
      # and change the queue service's image as well
      # - "--broker_api=http://guest:guest@queue:15672/api//"
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${TRAEFIK_PUBLIC_NETWORK?Variable not set}
        - traefik.constraint-label=${TRAEFIK_PUBLIC_TAG?Variable not set}
        - traefik.http.routers.${STACK_NAME?Variable not set}-flower-http.rule=Host(`flower.${DOMAIN?Variable not set}`)
        - traefik.http.routers.${STACK_NAME?Variable not set}-flower-http.entrypoints=http
        - traefik.http.routers.${STACK_NAME?Variable not set}-flower-http.middlewares=${STACK_NAME?Variable not set}-https-redirect
        - traefik.http.routers.${STACK_NAME?Variable not set}-flower-https.rule=Host(`flower.${DOMAIN?Variable not set}`)
        - traefik.http.routers.${STACK_NAME?Variable not set}-flower-https.entrypoints=https
        - traefik.http.routers.${STACK_NAME?Variable not set}-flower-https.tls=true
        - traefik.http.routers.${STACK_NAME?Variable not set}-flower-https.tls.certresolver=le
        - traefik.http.services.${STACK_NAME?Variable not set}-flower.loadbalancer.server.port=5555

  backend:
    image: '${DOCKER_IMAGE_BACKEND?Variable not set}:${TAG-latest}'
    #depends_on:
    #  - db
    env_file:
      - .env
    environment:
      - SERVER_NAME=${DOMAIN?Variable not set}
      - SERVER_HOST=https://${DOMAIN?Variable not set}
      # Allow explicit env var override for tests
      - SMTP_HOST=${SMTP_HOST}
    build:
      context: ./backend
      dockerfile: backend.dockerfile
      args:
        INSTALL_DEV: ${INSTALL_DEV-false}
    deploy:
      labels:
        - traefik.enable=true
        - traefik.constraint-label-stack=${TRAEFIK_TAG?Variable not set}
        - traefik.http.routers.${STACK_NAME?Variable not set}-backend-http.rule=PathPrefix(`/api`) || PathPrefix(`/docs`) || PathPrefix(`/redoc`) || PathPrefix(`/ws`)
        - traefik.http.services.${STACK_NAME?Variable not set}-backend.loadbalancer.server.port=80
        - traefik.http.routers.${STACK_NAME?Variable not set}-backend-https.tls=true  
        - traefik.http.routers.${STACK_NAME?Variable not set}-backend-https.tls.certresolver=le
        #- traefik.frontend.rule=PathPrefix:/api,/docs,/redoc,/ws
        #- traefik.port=80
        #- traefik.tags=${TRAEFIK_TAG}

  # Add the strapi headless CMS
  strapi:
    image: strapi/strapi
    env_file:
      - .env
    networks:
      - ${TRAEFIK_PUBLIC_NETWORK?Variable not set}    
      - default
    volumes:
      - ./cms:/srv/app
    #ports:
    #  - '1337:1337'
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${TRAEFIK_PUBLIC_NETWORK?Variable not set}
        - traefik.constraint-label=${TRAEFIK_PUBLIC_TAG?Variable not set}
        - traefik.http.routers.${STACK_NAME?Variable not set}-strapi-http.rule=Host(`strapi.${DOMAIN?Variable not set}`)
        - traefik.http.routers.${STACK_NAME?Variable not set}-strapi-http.entrypoints=http
        - traefik.http.routers.${STACK_NAME?Variable not set}-strapi-http.middlewares=${STACK_NAME?Variable not set}-https-redirect
        - traefik.http.routers.${STACK_NAME?Variable not set}-strapi-https.rule=Host(`strapi.${DOMAIN?Variable not set}`)
        - traefik.http.routers.${STACK_NAME?Variable not set}-strapi-https.entrypoints=https
        - traefik.http.routers.${STACK_NAME?Variable not set}-strapi-https.tls=true
        - traefik.http.routers.${STACK_NAME?Variable not set}-strapi-https.tls.certresolver=le
        - traefik.http.services.${STACK_NAME?Variable not set}-strapi.loadbalancer.server.port=1337

  celeryworker:
    image: '${DOCKER_IMAGE_CELERYWORKER?Variable not set}:${TAG-latest}'
    #depends_on:
    #  - db
    #  - queue
    env_file:
      - .env
    environment:
      - SERVER_NAME=${DOMAIN?Variable not set}
      - SERVER_HOST=https://${DOMAIN?Variable not set}
      # Allow explicit env var override for tests
      - SMTP_HOST=${SMTP_HOST?Variable not set}
    build:
      context: ./backend
      dockerfile: celeryworker.dockerfile
      args:
        INSTALL_DEV: ${INSTALL_DEV-false}

  frontend:
    image: '${DOCKER_IMAGE_FRONTEND?Variable not set}:${TAG-latest}'
    networks:
      - ${TRAEFIK_PUBLIC_NETWORK?Variable not set}
      - default    
    build:
      context: ./frontend
      args:
        FRONTEND_ENV: ${FRONTEND_ENV-production}
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=${TRAEFIK_PUBLIC_NETWORK?Variable not set}
        - traefik.constraint-label-stack=${TRAEFIK_TAG?Variable not set}
        - traefik.http.routers.${STACK_NAME?Variable not set}-frontend-http.rule=PathPrefix(`/`) || PathPrefix(`/ws`)
        - traefik.http.services.${STACK_NAME?Variable not set}-frontend.loadbalancer.server.port=80
        - traefik.http.routers.${STACK_NAME?Variable not set}-frontend-https.tls=true
        - traefik.http.routers.${STACK_NAME?Variable not set}-frontend-https.tls.certresolver=le

volumes:
  app-db-data:

networks:
  traefik-public:
    # Allow setting it to false for testing
    external: ${TRAEFIK_PUBLIC_NETWORK_IS_EXTERNAL-true}

There is probably something wrong with Traefik but my knowledge is very limited on that side I have to say and I haven't seen anything in the documentation. Did you manage to make the frontend call the api over HTTPS without any issue?

[abrichr]

Can you paste the code which issues this request?

[jceyrac]

Can you paste the code which issues this request?

Sure! In the frontend, the call is initiated in the method "mounted" from a component verva/frontend/src/components/CoinList.vue:

public async fetchPriceData(){
    this.setPairList();
    await dispatchGetPrices(this.$store, {pairs: this.pairs});
    this.dispatchPrices(readPrices(this.$store));
  }

dispatchGetPrices calls the following action in the store verva/frontend/src/store/coinInfos/actions:

export const actions = {
  async actionGetPrices(context: MainContext, payload: { pairs: IPair[] }) {
    console.log("inActionGetPrices...")
    try {
        const response = await api.getPrices(context.rootState.main.token,payload.pairs)
          .then(function (response){
            commitSetPrices(context, response.data);
            console.log("response ="+response.data)
          })
    } catch (error) {
        dispatchCheckApiError(context, error);
    }
  }

the getPrices method is defined in verva/frontend/src/api.ts:

async getPrices(token: string, data: IPair[]) {
    return await axios.put<IPrice[]>(`${apiUrl}/api/v1/prices`, data, authHeaders(token));
  }

The sources are here: https://gitlab.com/Ceyrac/verva

I just realized that I can login to the default admin interface https://verva.fr/login without any issue, in particular the "Blocked loading mixed active content" one:

This makes me think as you suggest, that the issue is not Traefik related but more code related. I'll dig into that.

[abrichr]

I had a similar issue where only one API call was being issued for HTTP for some reason, where all the other ones were being issued over HTTPS. I noticed that this was the only one which didn't have a trailing / at the end of the URI, and for some reason adding the trailing slash fixed it. Your URI also does not have a trailing slash:

`${apiUrl}/api/v1/prices`

Try changing this to the following and see if that helps:

`${apiUrl}/api/v1/prices/`

@tiangolo any idea why this would happen?

[jceyrac]

Thanks for your reply @abrichr, much appreciated. I added the trailing at the end of the URI but I still have the Mixed active content issue unfortunately: The page took longer than usual to load though... Maybe there are some logs to check somewhere?

[jceyrac]

I found the issue! In my code I'm calling the strapi CMS endpoint with the URI https://verva.fr:1337/coins whereas traefik changes it to https://strapi.verva.fr/coins:

 - traefik.http.routers.${STACK_NAME?Variable not set}-strapi-http.rule=Host(`strapi.${DOMAIN?Variable not set}`) 

I need to find a way to either keep https://verva.fr:1337/coins working in prod or https://strapi.verva.fr/coins in dev...

UPDATE: after adding the configuration for a new ApiUrlCms for strapi in the /frontend/src/env.ts it works all good. So I now have a production environment working, I try to setup the staging one. Thanks so much for sharing your deployment steps and help @abrichr !!!

[luanjubica-helius]

Hello, I have the following problem I have all my services running in my droplet. Like :

and traefik stack running the public traefik like :

I can access the public traefik dashboard... but my app is not exposed and I get a 404 on port 80.

Any advice?

[luanjubica-helius]

Nevermind, solved by getting the latest version of docker-compose.yml.

[tybirk-mindway]

Just want to chip in that in my situation it was the exact opposite of what abhichr described. Had the same mixed content errors where some calls were seemingly issued via HTTP for no apparent reason, and the fix was to replace

${apiUrl}/api/v1/prices/

with

${apiUrl}/api/v1/prices

In the axios call.

[thomas-chauvet]

Hi,

I don't really understand how to configure the part with CNAME:

# Configure your DNS, e.g.:
# CNAME stag ec2-...amazonaws.com.
# CNAME traefik stag.foo.com

I am running the app on AWS EC2 instance and my domain has been bought from OVH. I don't find where to add this CNAME part on OVH website.

Thanks a lot!

[abrichr]

This is outside the scope of this project, and I am unfamiliar with OVH, but a quick Google search led me to https://docs.ovh.com/ca/en/domains/web_hosting_how_to_edit_my_dns_zone/

On Mon, Mar 1, 2021 at 5:48 AM Thomas notifications@github.com wrote:

Hi,

I don't really understand how to configure the part with CNAME:

Configure your DNS, e.g.:

CNAME stag ec2-...amazonaws.com.

CNAME traefik stag.foo.com

I am running the app on AWS EC2 instance and my domain has been bought from OVH. I don't find where to add this CNAME part on OVH website.

Thanks a lot!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tiangolo/full-stack-fastapi-postgresql/issues/322#issuecomment-787851309, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF5DVZPTBUCYINLWJZN63LTBNWHBANCNFSM4T6Y4THQ .

[pramadito]

hey, thanks for writing this guide!

i want to write guide and making thesis to deploy full stack website using this reference

but first what configuration do you use to launch EC2?

i am currently trying to use your guide to launch production environment but have no luck for it to work and now troubleshooting one by one.

right now i am hosting it in :

• mudahbangun.com
• https://stag.mudahbangun.com/
• https://traefik.stag.mudahbangun.com/

[abrichr]

• AMI: Ubuntu Server 20.04 LTS 64-bit x86
• Type: t2.xlarge
• Storage: 256GiB SSD
• Security Group: All traffic (ideally you should only allow connections from a bastion host but I haven't set this up yet)

Everything else is default.

Hope this helps! Please post the article here when it's up 👍

[pramadito]

• AMI: Ubuntu Server 20.04 LTS 64-bit x86
• Type: t2.xlarge
• Storage: 256GiB SSD
• Security Group: All traffic (ideally you should only allow connections from a bastion host but I haven't set this up yet)

Everything else is default.

Hope this helps! Please post the article here when it's up 👍

thanks for the reply! i guess i will try again with bigger type, currently using small one

[abrichr]

If you describe the issue you are experiencing maybe I or someone else can help

On Wed, Mar 10, 2021 at 12:12 PM pramadito @.***> wrote:

• AMI: Ubuntu Server 20.04 LTS 64-bit x86
• Type: t2.xlarge
• Storage: 256GiB SSD
• Security Group: All traffic (ideally you should only allow connections from a bastion host but I haven't set this up yet)

Everything else is default.

Hope this helps! Please post the article here when it's up 👍

thanks for the reply! i guess i will try again with bigger type, currently using small one

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tiangolo/full-stack-fastapi-postgresql/issues/322#issuecomment-795757877, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF5DVYVOQLINXPYGZPZJFDTC6R7PANCNFSM4T6Y4THQ .

[pramadito]

If you describe the issue you are experiencing maybe I or someone else can help … On Wed, Mar 10, 2021 at 12:12 PM pramadito @.***> wrote: - AMI: Ubuntu Server 20.04 LTS 64-bit x86 - Type: t2.xlarge - Storage: 256GiB SSD - Security Group: All traffic (ideally you should only allow connections from a bastion host but I haven't set this up yet) Everything else is default. Hope this helps! Please post the article here when it's up 👍 thanks for the reply! i guess i will try again with bigger type, currently using small one — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#322 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF5DVYVOQLINXPYGZPZJFDTC6R7PANCNFSM4T6Y4THQ .

the issue currently i have is

• failed build after step 9(?). it freezes while doing that step -> i found it maybe t2.micro too small
• i cannot connect to ec2 using route 53

[pramadito]

i get everything working except " Unable to obtain ACME certificate for domains"

full error

root@ip-172-31-17-41:~/mudahbangun# docker service logs traefik_traefik -f
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | time="2021-03-12T07:42:40Z" level=info msg="Configuration loaded from flags."
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | time="2021-03-12T07:43:07Z" level=error msg="Unable to obtain ACME certificate for domains \"traefik.stag.mudahbangun.com\": unable to generate a certificate for the domains [traefik.stag.mudahbangun.com]: error: one or more domains had a problem:\n[traefik.stag.mudahbangun.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for traefik.stag.mudahbangun.com - check that a DNS record exists for this domain, url: \n" providerName=le.acme routerName=traefik-public-https@docker rule="Host(`traefik.stag.mudahbangun.com`)"
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | 10.0.0.2 - - [12/Mar/2021:07:54:57 +0000] "GET / HTTP/1.1" 404 19 "-" "-" 1 "-" "-" 0ms
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | 10.0.0.2 - - [12/Mar/2021:07:54:57 +0000] "GET /favicon.ico HTTP/1.1" 404 19 "-" "-" 2 "-" "-" 0ms
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | 10.0.0.2 - - [12/Mar/2021:07:55:08 +0000] "GET / HTTP/1.1" 404 19 "-" "-" 3 "-" "-" 0ms
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | 10.0.0.2 - - [12/Mar/2021:07:55:08 +0000] "GET /favicon.ico HTTP/1.1" 404 19 "-" "-" 4 "-" "-" 0ms
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | time="2021-03-12T07:57:44Z" level=error msg="Unable to obtain ACME certificate for domains \"flower.stag.mudahbangun.com\": unable to generate a certificate for the domains [flower.stag.mudahbangun.com]: error: one or more domains had a problem:\n[flower.stag.mudahbangun.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for flower.stag.mudahbangun.com - check that a DNS record exists for this domain, url: \n" routerName=stag-mudahbangun-com-flower-https@docker rule="Host(`flower.stag.mudahbangun.com`)" providerName=le.acme
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | time="2021-03-12T07:57:44Z" level=error msg="Unable to obtain ACME certificate for domains \"pgadmin.stag.mudahbangun.com\": unable to generate a certificate for the domains [pgadmin.stag.mudahbangun.com]: error: one or more domains had a problem:\n[pgadmin.stag.mudahbangun.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for pgadmin.stag.mudahbangun.com - check that a DNS record exists for this domain, url: \n" providerName=le.acme routerName=stag-mudahbangun-com-pgadmin-https@docker rule="Host(`pgadmin.stag.mudahbangun.com`)"
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | time="2021-03-12T07:57:44Z" level=error msg="Unable to obtain ACME certificate for domains \"traefik.stag.mudahbangun.com\": unable to generate a certificate for the domains [traefik.stag.mudahbangun.com]: error: one or more domains had a problem:\n[traefik.stag.mudahbangun.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for traefik.stag.mudahbangun.com - check that a DNS record exists for this domain, url: \n" routerName=traefik-public-https@docker rule="Host(`traefik.stag.mudahbangun.com`)" providerName=le.acme
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | 10.0.0.2 - - [12/Mar/2021:07:57:48 +0000] "GET / HTTP/1.1" 302 5 "-" "-" 5 "stag-mudahbangun-com-proxy-http@docker" "-" 0ms
traefik_traefik.1.uz5rjqr4g1v4@stag.mudahbangun.com    | time="2021-03-12T07:58:03Z" level=error msg="Unable to obtain ACME certificate for domains \"stag.mudahbangun.com,www.stag.mudahbangun.com\": unable to generate a certificate for the domains [stag.mudahbangun.com www.stag.mudahbangun.com]: error: one or more domains had a problem:\n[stag.mudahbangun.com] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Timeout during connect (likely firewall problem), url: \n[www.stag.mudahbangun.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for www.stag.mudahbangun.com - check that a DNS record exists for this domain, url: \n" providerName=le.acme routerName=stag-mudahbangun-com-proxy-https@docker rule="Host(`stag.mudahbangun.com`) || Host(`www.stag.mudahbangun.com`)" 

docker image already running

[thomas-chauvet]

I guess it's a problem with DNS records. Did you enter A rules properly where you bought your domain name?

It should look like:

• type of DNS record A - hostname: *.stag.mudahbangun.com - VM/Instance IP: 139.59.134.103
• type of DNS record A - hostname: stag.mudahbangun.com - VM/Instance IP: 139.59.134.103

Personnaly I use digital ocean DNS and it's pretty easy. I don't know how to do it on AWS.

According this DNS checker https://www.nexcess.net/web-tools/dns-checker/. It looks like you have properly configured stag.mudahbangun.com but there's nothing for the rule with the *.

I guess the issue comes from here.

[pramadito]

I guess it's a problem with DNS records. Did you enter A rules properly where you bought your domain name?

It should look like:

• type of DNS record A - hostname: *.stag.mudahbangun.com - VM/Instance IP: 139.59.134.103
• type of DNS record A - hostname: stag.mudahbangun.com - VM/Instance IP: 139.59.134.103

Personnaly I use digital ocean DNS and it's pretty easy. I don't know how to do it on AWS.

According this DNS checker https://www.nexcess.net/web-tools/dns-checker/. It looks like you have properly configured stag.mudahbangun.com but there's nothing for the rule with the *.

I guess the issue comes from here.

here's what i set up for route 53

i try to buy the domain from route 53, i guess i should use digital ocean next time

[thomas-chauvet]

I think you can create a new record with the same rule as stag.mudahbangun.com but instead of stag.mudahbangun.com change it by *.stag.mudahbangun.com (keep the rule with `stag.mudahbangun.com).

If you can't or it doesn't work you can try to create a record with aA rule instead of a CNAME with *.stag.mudahbangun.com, type A, and in "value/Route traffic to" you put the IP of the machine

[pramadito]

ok some good news i can access it on :

http://ec2-18-136-120-198.ap-southeast-1.compute.amazonaws.com/

and 18.136.120.198

but the bad news is it vomit 404 error

also am i doing it right like this?

[thomas-chauvet]

I didn't see it at first sight but I think you can remove the rediction from traefik.mudahbangun.com to stag.mudahbangun.com. I don't think the issue come from here but it could make things messy.

If the A rule doesn't work (according the DNS chhecker the rule has been taken into account) try to remove it and put a CNAME rule like the one with stag.mudahbangun.com but for *.stag.mudahbangun.com.

I am also seeing that in your docker ps screenshot you don't have exposed port for traefik. According docker-compose on dockerswarm.rocks port 80 and 443 should be exposed.

[pramadito]

I didn't see it at first sight but I think you can remove the rediction from traefik.mudahbangun.com to stag.mudahbangun.com. I don't think the issue come from here but it could make things messy.

If the A rule doesn't work (according the DNS chhecker the rule has been taken into account) try to remove it and put a CNAME rule like the one with stag.mudahbangun.com but for *.stag.mudahbangun.com.

I am also seeing that in your docker ps screenshot you don't have exposed port for traefik. According docker-compose on dockerswarm.rocks port 80 and 443 should be exposed.

Ok more good news i can access my traefik network, even tho cert is not good

[pramadito]

Good news, now it's working properly

certificate error can be fixed by rebooting EC2 again after it's been deployed

i guess i need to set port 80 and 443 open first before this works

[wangxr14]

@pramadito Hi can you please share what changes have you made before you can access traefik? I'm having the same problem that I just fixed certificate error by setting dns and got the containers running, but still cannot access traefik ui, and got 404 error when try to access my app

[pramadito]

@pramadito Hi can you please share what changes have you made before you can access traefik? I'm having the same problem that I just fixed certificate error by setting dns and got the containers running, but still cannot access traefik ui, and got 404 error when try to access my app

@wangxr14 i opened port 80 and 443 and set route 53 to stag.mudahbangun.con and *.stag.mudahbangun.com

More detail:

open port 80 and 443 in this docker security group port port:

and set rout stag.mudahbangun.com and *.stag.mudahbangun.com

everything else i do it the same

[pramadito]

More Update:

the minimal requirement in Amazon EC2 for deploying this Project Template is t2.Small with at least 10 gb of space

the free t2.micro is sadly too small can cause bash scripts/build.sh to crash and the database not working properly

if you want to see it in action.

https://stag.mudahbangun.com/

to login:

Username: test@test.com

password: pramadito1234

[ghost]

Hi ! Thanks a lot for the detailed post for the deployment. I have my traefik service running and my api is in https without problem. But I cannot access Flower and PGAdmin https://pgadmin.mifarmacia.app/ https://flower.mifarmacia.app/ with NET::ERR_CERT_AUTHORITY_INVALID even tho I declared in my DNS record the two subdomain with CNAME the server I'm running on ... Any idea ?

[pramadito]

Hi ! Thanks a lot for the detailed post for the deployment. I have my traefik service running and my api is in https without problem. But I cannot access Flower and PGAdmin https://pgadmin.mifarmacia.app/ https://flower.mifarmacia.app/ with NET::ERR_CERT_AUTHORITY_INVALID even tho I declared in my DNS record the two subdomain with CNAME the server I'm running on ... Any idea ?

try to restart amazon EC2, i don't have any problem with it so far

[ghost]

I'm not using an amazon server I'm depoying on a VPS instance of 1&1 ionos server

[pramadito]

i wonder why when i run

When changes are pushed to your repo, the following seems to be required in order to deploy:

git stash
git pull
git stash pop
bash scripts/build.sh
bash scripts/deploy.sh
docker service update --force $ENV_NAME_SHORT-${BASE_NAME}_backend -d
docker service update --force $ENV_NAME_SHORT-${BASE_NAME}_frontend -d

my image don't update correctly sometimes

Edit: nvm i found you have to do

bash scripts/build-push.sh

after build.sh to docker registry so it can update correctly

but if you don't have huge amazon ec2 you can do

 bash scripts/build.sh
 bash scripts/build-push.sh 

in local first

this way when you use deploy.sh the image will update correctly because you have push it to docker hub

[ghost]

Hello, I tried to put the system online and it worked well but now I have a lot of changes to apply and that's why I wanted to make a fresh start by deleting the images and stopping all the containers. But for some reason, the container keeps on starting over and over with new ids. I tried "docker stop" "docker container rm -f", to update the restart policy to "unless-stopped" and to "no" but nothing keeps the containers from restarting do you have any insights?

[abrichr]

There may be a bug in the code that your container is running that is causing the container to crash; try checking the logs.

[ghost]

Maybe I explained it badly but my container is not crashing. I just want to stop it to delete the db volume but it keeps on restarting everytime I use the stop/kill or rm command.

[gaganpreet]

When you deploy a stack using docker swarm, killing the container will usually restart the associated service. If you want to start over, you'd need to find the corresponding stack using docker stack ls then remove with docker stack rm.

[ghost]

After running for a while without any trouble the backend side of the app is not working anymore stag.mifarmacia.app I have either a 404 or a 502 error in the browser and this is the logs of the backend. Do you have any insights? ERROR:main:(psycopg2.OperationalError) could not translate host name "db" to address: Name or service not known

[anouar-zh]

Why would you install this stack on Ubuntu while you could deploy the docker images directly on AWS?

[wangxr14]

Hi, I got my service online successfully and was adding some changes, but after I followed the steps to deploy the updates, I found out backend image is not running, and api returns 404. Everything else looks fine, didn't see any error when running bash scripts/build.sh and bash scripts/deploy.sh, and I can see my changes to login page in frontend was successfully deployed. And things were all fine with docker-compose up locally. Anyone has idea what might cause this issue or how can I run that single image up?

[ghost]

Why would you install this stack on Ubuntu while you could deploy the docker images directly on AWS?

I'm working with ionos.es 1&1 where they have my domain and other elements so I thought it was more convenient. But do you have any insight into why I could have this particular error? (cause everything was working fine until this happened...)

[pramadito]

for future who read this deploying in AWS,

you need at least t2.small to run this operation in Amazon EC2

lower than that can cause some container not working properly

i learn you can push your build to docker hub

using

bash ./scripts/build-push.sh

then use

bash scripts/deploy.sh

so the EC2 can pull it from docker hub automatically, after you build it on your local.

it's gonna take a while to update it, around 5 minutes for me

[dabslants]

Thanks for this walk through. Is it possible to limit pgadmin & flower subdomain access? Maybe by applying the traefik admin auth middleware? if not then I'll have to remove both packages from public access. These packages should not be expose on a production deploy.

[petioptrv]

I feel like I should leave this comment here, but what had me confused for a very long time is that I thought the traefik.yml downloaded in this writeup is actually an improvement of the one included in the docker-compose.yml of this project, so I tried replacing the configs in docker-compose.yml. This is wrong. At the end of the day, you will have two traefik services: one is traefik as downloaded and ran in this writeup, and the other is proxy from the project's docker-compose.yml.