multiple subdomains question

[jeremeybingham]

Been struggling all day with getting this setup to work with two different subdomains which both need to accept https traffic (for a flask site/development site for same) - need to see if anyone has any insight. I'm getting the following errors when the first challenge is attempted while running the script, with some modifications.

On the first cert verification attempt:

Waiting for verification...
Challenge failed for domain my.example.domain
http-01 challenge for my.example.domain
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: my.example.domain
   Type:   connection
   Detail: Fetching
   http://my.example.domain/.well-known/acme-challenge/Z*****************0:
   Connection refused

    ... blah blah A records, etc ...

### Reloading nginx ...
cannot exec in a stopped state: unknown

In docker-compose up output - the reason nginx fails above, presumably?

nginx_1 | <timestamp-here> [emerg] 1#1: open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/app.conf:22

nginx_1 | nginx: [emerg] open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/app.conf:22

My Desired Setup:

• NGINX reverse proxy to accept/redirect all http/s traffic to SSL to both: "my.example.com" and "my-dev.example.com"
• two basically identical containers, "web" and "web-dev", which are flask apps running on 5000:80 and 5001:80.
• I want to bring it all up at once with the script and then docker-compose up, issuing separate certs to each (I'm fine with changing it to a single cert for both if that's possible/easier for some reason)

I've tried everything I could find from searching anything remotely related to the issue and suspect it's more a fundamental "not understanding NGINX config" issue on my part than a bug, but I hope someone can help and this question can be a reference for anyone else similarly stuck. I think there might be a solution in the nginx.conf file, having to do with adding some upstream entries, possibly, but I don't quite understand where to begin.

All my code is below, thank you in advance!

my nginx.conf

server {
    listen 80;
    server_name my.example.domain;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name my.example.domain;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/my.example.domain/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/my.example.domain/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass  http://web:5000;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }
}

server {
    listen 80;
    server_name my-dev.example.domain;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name my-dev.example.domain;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/my-dev.example.domain/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/my-dev.example.domain/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass  http://web-dev:5001;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }
}

---

my init-letsencrypt.sh

#!/bin/bash

domains=(my.example.domain my-dev.example.domain)
rsa_key_size=4096
data_path="./data/certbot"
email="myexample@email.address" # Adding a valid address is strongly recommended
staging=1 # Set to 1 if you're testing your setup to avoid hitting request limits

if [ -d "$data_path" ]; then
  read -p "Existing data found for $domains. Continue and replace existing certificate? (y/N) " decision
  if [ "$decision" != "Y" ] && [ "$decision" != "y" ]; then
    exit
  fi
fi

if [ ! -e "$data_path/conf/options-ssl-nginx.conf" ] || [ ! -e "$data_path/conf/ssl-dhparams.pem" ]; then
  echo "### Downloading recommended TLS parameters ..."
  mkdir -p "$data_path/conf"
  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx.conf > "$data_path/conf/options-ssl-nginx.conf"
  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot/ssl-dhparams.pem > "$data_path/conf/ssl-dhparams.pem"
  echo
fi

echo "### Creating dummy certificate for $domains ..."
path="/etc/letsencrypt/live/$domains"
mkdir -p "$data_path/conf/live/$domains"
docker-compose run --rm --entrypoint "\
  openssl req -x509 -nodes -newkey rsa:1024 -days 1\
    -keyout '$path/privkey.pem' \
    -out '$path/fullchain.pem' \
    -subj '/CN=localhost'" certbot
echo

echo "### Starting nginx ..."
docker-compose up --force-recreate -d nginx #tried this both ways
echo

echo "### Deleting dummy certificate for $domains ..."
docker-compose run --rm --entrypoint "\
  rm -Rf /etc/letsencrypt/live/$domains && \
  rm -Rf /etc/letsencrypt/archive/$domains && \
  rm -Rf /etc/letsencrypt/renewal/$domains.conf" certbot
echo

echo "### Requesting Let's Encrypt certificate for $domains ..."
#Join $domains to -d args
domain_args=""
for domain in "${domains[@]}"; do
  domain_args="$domain_args -d $domain"
done

# Select appropriate email arg
case "$email" in
  "") email_arg="--register-unsafely-without-email" ;;
  *) email_arg="--email $email" ;;
esac

# Enable staging mode if needed
if [ $staging != "0" ]; then staging_arg="--staging"; fi

docker-compose run --rm --entrypoint "\
  certbot certonly --webroot -w /var/www/certbot \
    $staging_arg \
    $email_arg \
    $domain_args \
    --rsa-key-size $rsa_key_size \
    --agree-tos \
    --eff-email \  ##"I added this and it should have no weird effect"
    --force-renewal" certbot
echo

echo "### Reloading nginx ..."
docker-compose exec nginx nginx -s reload

---

my docker-compose

version: '3'

services:
  nginx:
    image: nginx:stable-alpine
    restart: unless-stopped
    volumes:
      - ./data/nginx:/etc/nginx/conf.d
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
      - ./data/nginx/file_size.conf:/etc/nginx/file_size.conf # I added this, unrelated?
    ports:
      - "80:80"
      - "443:443"
    links:
      - web
      - web-dev
    command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"

  certbot:
    image: certbot/certbot
    restart: unless-stopped
    volumes:
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"

  web:
    build:
      context: .
      dockerfile: prod.Dockerfile
    restart: unless-stopped
    volumes:
      - ./data/web:/data/web
    ports:
      - "5000:80"

  web-dev:
    build:
      context: .
      dockerfile: dev.Dockerfile
    restart: unless-stopped
    volumes:
      - ./data/web-dev:/data/web-dev
    ports:
      - "5001:80"
    environment:
      FLASK_ENV: development

---

my 'web' container dockerfile

FROM python:3.7-slim-buster

# handle static files
ENV STATIC_URL /static

# if making changes this should be an absolute path
ENV STATIC_PATH /data/web/static

# set working directory and import /data/web/*
COPY ./data/web /data/web
WORKDIR /data/web

# Make /data/web* available to be imported by Python globally
ENV PYTHONPATH=/data/web

ENV FLASK_APP /data/web/app.py
ENV FLASK_RUN_HOST 0.0.0.0

# install requirements 
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .
CMD ["flask", "run"]

---

my 'web-dev' container dockerfile

FROM python:3.7-slim-buster

# handle static files
ENV STATIC_URL /static

# if making changes this should be an absolute path
ENV STATIC_PATH /data/web-dev/static

# set working directory and import /data/web-dev/*
COPY ./data/web-dev /data/web-dev
WORKDIR /data-dev/web

# Make /data/web-dev* available to be imported by Python globally
ENV PYTHONPATH=/data/web-dev

ENV FLASK_APP /data/web-dev/app.py
ENV FLASK_RUN_HOST 0.0.0.0

# install requirements 
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .
CMD ["flask", "run"]

[chandruk4321]

I think you ran into the same issue as I had yesterday. First of all make sure your subdomains are reachable (try pinging them). I followed the steps below to get the current master code working for 2 domains.

For multiple domains

1. Modify configuration:

   • Add domains seperated by space to init-letsencrypt.sh
   • create new configuration file for every domain under nginx folder (similar to existing app.conf)
   • change the "server_name" in every configuration file to the respective domain name
   • NOTE: do not change the domain name in the certificate path. It should always be the first one you added to init-letsencrypt.sh

2. Run the init script: ./init-letsencrypt.sh

The mistake I did was to change all the occurances of example.org in the second domain's configuration file. The certificate path should be same for both configurations.

[jeremeybingham]

Weird, so I made that change noted exactly as you suspected:

The mistake I did was to change all the occurances of example.org in the second domain's configuration file. The certificate path should be same for both configurations.

And now the certificates are issued correctly when I run the script, but neither of my domains resolve to the right page - they just time out.

On this step:

create new configuration file for every domain under nginx folder (similar to existing app.conf)

I wasn't sure what to do - as above, I had the nginx.conf for both domains/servers in a single file. I split that file in half; naming the two resulting files "web.conf" and "web-dev.conf" and ran it both like that and with them in a single file as above - both methods seem to work in terms of issuing certificates but both also result in the timeout when I try to load the page in a browser.

However, that's still an nginx issue, not one with this script, I guess. I'll keep working on it and try to figure out what's going on - I suspect it's either the ports or the fact that I need to clear out my Docker containers and old images on this machine completely, maybe there's remnants of old port allocations or something causing a problem. THANK YOU!

[chandruk4321]

for your information I am using latest image of nginx, not the one in the master code. Here is my docker compose

version: '2'

services: nginx: image: nginx restart: unless-stopped volumes:

• ./data/nginx:/etc/nginx/conf.d
• ./data/certbot/conf:/etc/letsencrypt
• ./data/certbot/www:/var/www/certbot ports:
• "80:80"
• "443:443" command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'" network_mode: "host" certbot: image: certbot/certbot restart: unless-stopped volumes:
• ./data/certbot/conf:/etc/letsencrypt
• ./data/certbot/www:/var/www/certbot entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"

[wmnnd]

This problem should now be a thing of the past! @mansard, @kannach: Please check out the 2.x branch of this repository. Multiple domains can now be configured simply by calling ./init-letsencrypt.sh -d domain1.org -d domain2.org.