search

nginx / njs-acme

Nginx NJS module runtime to work with ACME providers like Let's Encrypt for automated no-reload TLS certificate issue/renewal.
Apache License 2.0
66 stars 11 forks source link

I'm Helpless, what am I doing wrong? #44

Closed engAmirEng closed 10 months ago

engAmirEng commented 10 months ago

I followed exactly like the guide, but here is the result, https://mytest2.nilu.website/ what is wrong? I'm helpless

zsteinkamp commented 10 months ago

Hi @engAmirEng -- Can you post your nginx.conf file?

engAmirEng commented 10 months ago

@zsteinkamp 

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;

load_module modules/ngx_http_js_module.so;

events {
    worker_connections  1024;
}

http {
    include             /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                                      '$status $body_bytes_sent "$http_referer" '
                                      '"$http_user_agent" "$http_x_forwarded_for"'
    access_log  main  /var/log/nginx/access.log  main;

    sendfile            on;

    keepalive_timeout    65;

    upstream django_upstream {
        server django:5000;
        keepalive 4;
    }
    upstream flower_upstream {
        server flower:5555;
        keepalive 2;
    }

    js_path "/etc/nginx/njs_modules/";
    js_fetch_trusted_certificate /etc/ssl/certs/ISRG_Root_X1.pem;

    js_import acme from acme.js;

    resolver 8.8.8.8 8.8.4.4; # Google

    js_shared_dict_zone zone=acme:1m;

    server {
        listen       80;
        listen  443  ssl;
        server_name  mytest2.nilu.website;
        js_var $njs_acme_server_names mytest2.nilu.website;
        js_var $njs_acme_account_email amirsepahan78@gmail.com;

        js_var $njs_acme_shared_dict_zone_name acme;

        js_set $dynamic_ssl_cert acme.js_cert;
        js_set $dynamic_ssl_key acme.js_key;

        ssl_certificate data:$dynamic_ssl_cert;
        ssl_certificate_key data:$dynamic_ssl_key;

        location ~ "^/\.well-known/acme-challenge/[-_A-Za-z0-9]{22,128}$" {
            js_content acme.challengeResponse;
        }

        location = /acme/auto {
            js_content acme.clientAutoMode;
        }

        location /media/ {
            alias /usr/share/nginx/media/;
        }
        location /flower/ {
            proxy_redirect      off;
            proxy_set_header    X-Real-IP $remote_addr;
            proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header    Host $http_host;
            proxy_pass http://flower_upstream;
        }
        location / {
            proxy_redirect      off;
            proxy_set_header    X-Real-IP $remote_addr;
            proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header    Host $http_host;
            proxy_pass http://django_upstream;
        }
  }
}
zsteinkamp commented 10 months ago

Try removing or commenting out this line:

ssl_trusted_certificate  /etc/nginx/njs-acme/mytest2.nilu.website.csr

If that still doesn't work, perhaps send any nginx log lines that are emitted on server startup and when making a request.

engAmirEng commented 10 months ago

@zsteinkamp There is no ssl_trusted_certificate, I was just playing around with it, the current config does not have it

zsteinkamp commented 10 months ago

Ok ... How about the logs then?

engAmirEng commented 10 months ago

@zsteinkamp I'm running it inside docker compose, is that relative?

zsteinkamp commented 10 months ago

Assuming you mean "relevant" rather than "relative", I am running my own installation of this under docker compose, so I know it is a situation that works.

So please send your nginx log output, your docker-compose.yml file and the output of "docker ps" when you have nginx running.

engAmirEng commented 10 months ago

@zsteinkamp these are the ony log that are generated

django-stack-template-nginx-1  | 2023/12/31 23:41:50 [warn] 29#29: *2 js: njs-acme: [utils] error Error: No such file or directory while reading a private key from /etc/nginx/njs-acme/account_private_key.json
django-stack-template-nginx-1  | 2023/12/31 23:41:50 [error] 29#29: *2 js exception: Error: unhandled promise rejection: Error: No such file or directory, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 5.198.169.193 - - [31/Dec/2023:23:41:50 +0000] "GET /acme/auto HTTP/1.1" 500 579 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
django-stack-template-nginx-1  | 2023/12/31 23:42:22 [warn] 30#30: *1 js: njs-acme: [utils] error Error: No such file or directory while reading a private key from /etc/nginx/njs-acme/account_private_key.json
django-stack-template-nginx-1  | 2023/12/31 23:42:24 [crit] 30#30: *1 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 2023/12/31 23:42:27 [crit] 30#30: *1 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 2023/12/31 23:42:27 [crit] 30#30: *1 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 3.145.60.254 - - [31/Dec/2023:23:42:29 +0000] "GET /.well-known/acme-challenge/W30myi7GqLPv6cookXti2238GQBeZpwCLuKZILtbnoo HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
django-stack-template-nginx-1  | 66.133.109.36 - - [31/Dec/2023:23:42:29 +0000] "GET /.well-known/acme-challenge/W30myi7GqLPv6cookXti2238GQBeZpwCLuKZILtbnoo HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
django-stack-template-nginx-1  | 2023/12/31 23:42:30 [crit] 30#30: *1 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 5.198.169.193 - - [31/Dec/2023:23:42:34 +0000] "GET /acme/auto HTTP/1.1" 200 328 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
zsteinkamp commented 10 months ago

Ok this is informative. Can you send the other things I asked about? docker-compose.yml and the output of docker ps when you have the nginx container running.

engAmirEng commented 10 months ago

@zsteinkamp 


version: '3'

volumes:
  production_postgres_data: {}
  production_postgres_data_backups: {}
  production_django_media: {}

services:
  django: &django
    build:
      context: .
      dockerfile: ./compose/production/django/Dockerfile

    image: name_goes_here_production_django
    volumes:
      - production_django_media:/app/name_goes_here/media
    depends_on:
      - postgres
      - redis
    env_file:
      - ./.envs/.django
      - ./.envs/.postgres
    command: /start

  postgres:
    build:
      context: .
      dockerfile: ./compose/production/postgres/Dockerfile
    image: name_goes_here_production_postgres
    volumes:
      - production_postgres_data:/var/lib/postgresql/data
      - production_postgres_data_backups:/backups
    env_file:
      - ./.envs/.postgres

  redis:
    image: redis:6

  celeryworker:
    <<: *django
    image: name_goes_here_production_celeryworker
    command: /start-celeryworker

  celerybeat:
    <<: *django
    image: name_goes_here_production_celerybeat
    command: /start-celerybeat

  flower:
    <<: *django
    image: name_goes_here_production_flower
    command: /start-flower

  nginx:
    build:
      context: .
      dockerfile: ./compose/production/nginx/Dockerfile
    image: name_goes_here_local_nginx
    depends_on:
      - django
    volumes:
      - production_django_media:/usr/share/nginx/media:ro
    ports:
      - '0.0.0.0:80:80'
      - '0.0.0.0:443:443'
engAmirEng commented 10 months ago

@zsteinkamp 

FROM nginx:1.25
COPY ./compose/production/nginx/nginx.conf /etc/nginx/nginx.conf
COPY ./compose/production/nginx/njs-acme-ac656e1.js /etc/nginx/njs_modules/acme.js
engAmirEng commented 10 months ago

@zsteinkamp

the logs began to be written when I triggered /acme/auto


django-stack-template-nginx-1  | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
django-stack-template-nginx-1  | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
django-stack-template-nginx-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
django-stack-template-nginx-1  | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
django-stack-template-nginx-1  | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
django-stack-template-nginx-1  | /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
django-stack-template-nginx-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
django-stack-template-nginx-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
django-stack-template-nginx-1  | /docker-entrypoint.sh: Configuration complete; ready for start up
django-stack-template-nginx-1  | 2024/01/01 01:17:55 [notice] 1#1: using the "epoll" event method
django-stack-template-nginx-1  | 2024/01/01 01:17:55 [notice] 1#1: nginx/1.25.3
django-stack-template-nginx-1  | 2024/01/01 01:17:55 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) 
django-stack-template-nginx-1  | 2024/01/01 01:17:55 [notice] 1#1: OS: Linux 5.15.0-89-generic
django-stack-template-nginx-1  | 2024/01/01 01:17:55 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
django-stack-template-nginx-1  | 2024/01/01 01:17:55 [notice] 1#1: start worker processes
django-stack-template-nginx-1  | 2024/01/01 01:17:55 [notice] 1#1: start worker process 29
django-stack-template-nginx-1  | 2024/01/01 01:17:55 [notice] 1#1: start worker process 30
django-stack-template-nginx-1  | 2024/01/01 01:19:31 [error] 29#29: *1 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.crt . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:19:31 [error] 29#29: *1 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.key . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:19:31 [error] 29#29: *1 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 5.198.169.193, server: 0.0.0.0:443
django-stack-template-nginx-1  | 2024/01/01 01:19:31 [error] 30#30: *2 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.crt . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:19:31 [error] 30#30: *2 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.key . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:19:31 [error] 30#30: *2 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 5.198.169.193, server: 0.0.0.0:443
django-stack-template-nginx-1  | 5.198.169.193 - - [01/Jan/2024:01:19:46 +0000] "GET / HTTP/1.1" 404 179 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
django-stack-template-nginx-1  | 2024/01/01 01:20:02 [error] 29#29: *5 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.crt . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:20:02 [error] 29#29: *5 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.key . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:20:02 [error] 29#29: *5 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 5.198.169.193, server: 0.0.0.0:443
django-stack-template-nginx-1  | 2024/01/01 01:20:03 [error] 29#29: *6 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.crt . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:20:03 [error] 29#29: *6 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.key . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:20:03 [error] 29#29: *6 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 5.198.169.193, server: 0.0.0.0:443
django-stack-template-nginx-1  | 2024/01/01 01:20:04 [error] 29#29: *7 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.crt . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:20:04 [error] 29#29: *7 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.key . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:20:04 [error] 29#29: *7 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 5.198.169.193, server: 0.0.0.0:443
django-stack-template-nginx-1  | 2024/01/01 01:20:05 [error] 29#29: *8 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.crt . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:20:05 [error] 29#29: *8 js: njs-acme: error reading from file: /etc/nginx/njs-acme/mytest2.nilu.website.key . Error=Error: No such file or directory
django-stack-template-nginx-1  | 2024/01/01 01:20:05 [error] 29#29: *8 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 5.198.169.193, server: 0.0.0.0:443
django-stack-template-nginx-1  | 2024/01/01 01:20:05 [warn] 29#29: *3 js: njs-acme: [utils] error Error: No such file or directory while reading a private key from /etc/nginx/njs-acme/account_private_key.json
django-stack-template-nginx-1  | 2024/01/01 01:20:06 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 2024/01/01 01:20:07 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 2024/01/01 01:20:08 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 66.133.109.36 - - [01/Jan/2024:01:20:12 +0000] "GET /.well-known/acme-challenge/4n0QYMVTgVJrRWgpu1G2FAmQ8lVsIUoajI5zrHPZ31A HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
django-stack-template-nginx-1  | 2024/01/01 01:20:12 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 2024/01/01 01:20:16 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 3.131.94.117 - - [01/Jan/2024:01:20:22 +0000] "GET /.well-known/acme-challenge/4n0QYMVTgVJrRWgpu1G2FAmQ8lVsIUoajI5zrHPZ31A HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
django-stack-template-nginx-1  | 34.222.221.185 - - [01/Jan/2024:01:20:22 +0000] "GET /.well-known/acme-challenge/4n0QYMVTgVJrRWgpu1G2FAmQ8lVsIUoajI5zrHPZ31A HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
django-stack-template-nginx-1  | 2024/01/01 01:20:36 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 2024/01/01 01:20:38 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 2024/01/01 01:20:38 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 2024/01/01 01:20:41 [crit] 29#29: *3 connect() to [2606:4700:60:0:f41b:d4fe:4325:6026]:443 failed (99: Cannot assign requested address) while SSL handshaking to fetch target, client: 5.198.169.193, server: mytest2.nilu.website, request: "GET /acme/auto HTTP/1.1", host: "mytest2.nilu.website"
django-stack-template-nginx-1  | 5.198.169.193 - - [01/Jan/2024:01:20:42 +0000] "GET /acme/auto HTTP/1.1" 200 328 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
zsteinkamp commented 10 months ago

Hi @engAmirEng -- When I load your site, I see that there is a certificate there but it is issued by Let's Encrypt's Staging environment, which will not pass validation in browsers.

You will need to set a configuration variable with the correct value to point to the Let's Encrypt production environment. Additionally, you should add some content to your docker-compose.yml file to trigger the request to /acme/auto periodically. Now that I look at the README.md on the njs-acme repo, I can see that the production environment setting is not super clear, so I'll improve that.

So to fix things, there are two things you will need to add:

1) Add this line to your nginx.conf. This is the production Let's Encrypt:

js_var $njs_acme_directory_uri "https://acme-v02.api.letsencrypt.org/directory";

2) Add this to your docker-compose.yml in the nginx: service. This will request /acme/auto periodically so that you don't have to remember to do that before your certificate expires:

    healthcheck:
      test: ["CMD", "curl", "-f", "http://mytest2.nilu.website/acme/auto"]
      interval: 90s
      timeout: 90s
      retries: 3
      start_period: 10s

After adding the above and rebuilding/restarting the nginx container, you will need to remove the staging key/cert from the container. You can do that by getting a shell in the nginx container with:

docker compose exec nginx bash

Then run:

rm /etc/nginx/njs-acme/*.{crt,key}
exit

The next time you hit /acme/auto (or let the healthcheck run) then it will request the production (not staging) certificate.

Note that we have an upcoming release of njs-acme that eliminates the need for (2) above, but it will be a month or so before it's available.

engAmirEng commented 10 months ago

Wow, it worked, thanks, I almost lost it

zsteinkamp commented 10 months ago

Hurrah! Glad to hear it. Thanks for posting the troubles -- it will help us improve the documentation for future users. This is a brand new project, so we're all learning.

engAmirEng commented 10 months ago

@zsteinkamp By the way Is it a good idea to allocate a volume to /etc/ nginx/njs-acme ?

zsteinkamp commented 10 months ago

That is how I have it set up with my home installation... https://github.com/zsteinkamp/web-proxy/blob/main/docker-compose.yml The advantage of the volume is the cert/key will persist between rebuilds of the container. It's nice to have that option.