search

0xJacky / nginx-ui

Yet another WebUI for Nginx
https://nginxui.com
GNU Affero General Public License v3.0
2.22k stars 196 forks source link

容器化部署证书无法自动更新 #182

Closed hale177 closed 10 months ago

hale177 commented 10 months ago

环境： docke-compose 安装 版本： v1.9.9-4

现象： 我有多个站点domain ，证书通过 http-01 或者 dns-01 生成（有两个站点因为解析的内网ip，所有用了 dns-01 方式），但是发现很多证书都已过期或临近快过期并没有自动更新。

然后我进行了 compose stop 和 start 操作并观察了日志发现，启动的时候系统会去进行证书更新操作，但如果碰到第一个域名更新异常，整个更新过程就停止了，不管后续其他域名正常与否 。 日志如下：

ngzz  | 2023/11/05 01:24:04 [INFO] [k8s.xxx.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/280570705766
ngzz  | 2023/11/05 01:24:04 [INFO] [k8s.xxx.com] acme: Could not find solver for: tls-alpn-01
ngzz  | 2023/11/05 01:24:04 [INFO] [k8s.xxx.com] acme: use http-01 solver
ngzz  | 2023/11/05 01:24:04 [INFO] [k8s.xxx.com] acme: Trying to solve HTTP-01
ngzz  | 162.216.150.39 - - [05/Nov/2023:01:24:04 +0800] "GET / HTTP/1.1" 200 45 "-" "Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com" "-"
ngzz  | 2023-11-05 01:24:12 INFO cert/auto_cert.go:19 Auto Cert 2023/11/05 01:24:12 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/280570705766
ngzz  | 
ngzz  | 2023/11/05 01:24:12 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/280570705766
ngzz  | 2023-11-05 01:24:13 ERROR cert/auto_cert.go:34 AutoCert issue cert fail to obtain: error: one or more domains had a problem:
ngzz  | [k8s.xxx.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: no valid A records found for k8s.xxx.com; no valid AAAA records found for k8s.xxx.com

问题： 在反复测试N次后，发现了 2 个问题 1，证书逻辑是否需要优化下 ? 某个域名更新报错后 直接跳过，继续进行下一个域名的证书更新（因为测试时正好更新到了采用dns-01方式的内网ip 域名 就报错了，我手工处理后，后面就正常了） 2，通过观察日志发现一定要手工执行 docker-compose stop + start 才会触发证书更新 ， 而一直 running 的容器服务永远不会触发证书更新操作 。

0xJacky commented 10 months ago

好的，我确认一下

0xJacky commented 10 months ago

可以试试更新到 v2.0.0-beta x 的版本，之前这个问题应该已经修复过了

leic4u commented 10 months ago

我也是 docker compose 部署，版本v2.0.0-beta 4。

证书列表中有一个配置站点时申请的证书，应该是配置时自动就在证书列表也生成了一条数据。但是这个证书的自动更新是被禁用的，且没有找到手动开启的按钮。

请问下这个自动更新证书，是在到期前才会自动变成启用吗？

0xJacky commented 10 months ago

我也是 docker compose 部署，版本v2.0.0-beta 4。

证书列表中有一个配置站点时申请的证书，应该是配置时自动就在证书列表也生成了一条数据。但是这个证书的自动更新是被禁用的，且没有找到手动开启的按钮。

请问下这个自动更新证书，是在到期前才会自动变成启用吗？

截图个看看，正常是打开之后就会自动生成一条数据，并且他的自动更新也是打开的状态，或者你先删掉这条记录，重新尝试打开Server 里面申请证书的开关看看

leic4u commented 10 months ago

昨晚我没有删除证书的时候点了一下申请证书，没有报错，但申请一直卡着的，等了很久无奈只能关机睡觉了。

刚刚我去证书列表看，已经没有证书了，然后我又去申请证书，还是一直卡着的，也无法关闭，只能刷新页面。

image

另外我尝试新建站点，在申请证书时也会卡住。重启了容器也不行。

0xJacky commented 10 months ago

你是不是做了反向代理

leic4u commented 10 months ago

你是不是做了反向代理

现在有2个配置。

配置1：

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}
server {
    listen 80;
    listen [::]:80;
    server_name abc.xyz.org *.abc.xyz.org;
    if ($server_port = 80) {
        rewrite ^(/.*)$ https://$host$1 permanent;
    }
    location /.well-known/acme-challenge {
        proxy_set_header Host $host;
        proxy_set_header X-Real_IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr:$remote_port;
        proxy_pass http://127.0.0.1:9180;
    }
}
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name abc.xyz.org;
    ssl_certificate /etc/nginx/ssl/abc.xyz.org/fullchain.cer;
    ssl_certificate_key /etc/nginx/ssl/abc.xyz.org/private.key;
    http2 on;
    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_pass http://127.0.0.1:2053/;
    }
    location /.well-known/acme-challenge {
        proxy_set_header Host $host;
        proxy_set_header X-Real_IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr:$remote_port;
        proxy_pass http://127.0.0.1:9180;
    }
}

配置2：

server {
    listen 80;
    listen [::]:80;
    server_name 222.abc.xyz.org;
    location /.well-known/acme-challenge {
        proxy_set_header Host $host;
        proxy_set_header X-Real_IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr:$remote_port;
        proxy_pass http://127.0.0.1:9180;
    }
}
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    server_name 222.abc.xyz.org;
    location /.well-known/acme-challenge {
        proxy_set_header Host $host;
        proxy_set_header X-Real_IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr:$remote_port;
        proxy_pass http://127.0.0.1:9180;
    }
}
0xJacky commented 10 months ago

docker 是直接暴露的 80 和 443 端口到公网ip吗

0xJacky commented 10 months ago

这个情况是 Nginx UI 自己的 WebSocket 无法访问，一般是配置问题

leic4u commented 10 months ago

docker 是直接暴露的 80 和 443 端口到公网ip吗

下面是 docker compose 的配置，为了用在配置里能用 127.0.0.1 和 localhost 来写配置，所以我是直接用的 host 网络，这个会造成 WebSocket 问题么。我加下 WebSocket 的配置试试。

version: '3.3'
services:
    nginx-ui:
        stdin_open: true
        tty: true
        container_name: nginx-ui
        restart: always
        environment:
            - TZ=Asia/Shanghai
        volumes:
            - './nginx:/etc/nginx'
            - './nginx-ui:/etc/nginx-ui'
            - './var/www:/var/www'
        network_mode: host
        image: 'uozi/nginx-ui:latest'
0xJacky commented 10 months ago

docker 是直接暴露的 80 和 443 端口到公网ip吗

下面是 docker compose 的配置，为了用在配置里能用 127.0.0.1 和 localhost 来写配置，所以我是直接用的 host 网络，这个会造成 WebSocket 问题么。我加下 WebSocket 的配置试试。

version: '3.3'
services:
    nginx-ui:
        stdin_open: true
        tty: true
        container_name: nginx-ui
        restart: always
        environment:
            - TZ=Asia/Shanghai
        volumes:
            - './nginx:/etc/nginx'
            - './nginx-ui:/etc/nginx-ui'
            - './var/www:/var/www'
        network_mode: host
        image: 'uozi/nginx-ui:latest'

你可以 F12 看网络请求，他会新建一个 Websocket 链接的，如果连不上，那就会卡主，或者看看 Nginx UI docker 里的 logs 看看有无报错

leic4u commented 10 months ago

F12 的网络请求如下： image

nginx-ui 日志如下：

2023/11/11 16:27:21 /home/runner/work/nginx-ui/nginx-ui/server/query/chat_gpt_logs.gen.go:315
[0.164ms] [rows:1] SELECT * FROM `chat_gpt_logs` WHERE `chat_gpt_logs`.`name` = "/etc/nginx/sites-available/abc.xyz.org" ORDER BY `chat_gpt_logs`.`name` LIMIT 1

2023/11/11 16:27:21 /home/runner/work/nginx-ui/nginx-ui/server/query/sites.gen.go:323
[0.051ms] [rows:0] SELECT * FROM `sites` WHERE `sites`.`path` = "/etc/nginx/sites-available/abc.xyz.org" AND `sites`.`deleted_at` IS NULL ORDER BY `sites`.`id` LIMIT 1

2023/11/11 16:27:21 /home/runner/work/nginx-ui/nginx-ui/server/model/cert.go:31 record not found
[0.069ms] [rows:0] SELECT * FROM `certs` WHERE `certs`.`filename` = "abc.xyz.org" AND `certs`.`deleted_at` IS NULL ORDER BY `certs`.`id` LIMIT 1
2023-11-11 16:27:21     WARN    api/domain.go:129       cert record not found
[GIN] 2023/11/11 - 16:27:21 | 200 |    8.836444ms |   **.**.**.201 | GET      "/api/domain/abc.xyz.org"
**.**.**.201 - - [11/Nov/2023:16:27:21 +0800] "GET /api/domain/abc.xyz.org HTTP/1.1" 200 5087 "http://**.**.**.201:443/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "-"

2023/11/11 16:27:21 /home/runner/work/nginx-ui/nginx-ui/server/model/auth.go:56
[0.120ms] [rows:1] SELECT * FROM `auth_tokens` WHERE token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiaWNhb2xlaSIsImV4cCI6MTY5OTcxODk0OX0.-ONEes4SJ85InLzbWc75tusGPvmpXIsaiAZbx6oqaOA"
[GIN] 2023/11/11 - 16:27:21 | 200 |    1.292404ms |   **.**.**.201 | GET      "/api/template/blocks"
**.**.**.201 - - [11/Nov/2023:16:27:21 +0800] "GET /api/template/blocks HTTP/1.1" 200 2603 "http://**.**.**.201:443/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "-"

2023/11/11 16:27:28 /home/runner/work/nginx-ui/nginx-ui/server/model/auth.go:56
[0.124ms] [rows:1] SELECT * FROM `auth_tokens` WHERE token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiaWNhb2xlaSIsImV4cCI6MTY5OTcxODk0OX0.-ONEes4SJ85InLzbWc75tusGPvmpXIsaiAZbx6oqaOA"
[GIN] 2023/11/11 - 16:27:28 | 200 |    1.209438ms |   **.**.**.201 | GET      "/api/template/blocks"
**.**.**.201 - - [11/Nov/2023:16:27:28 +0800] "GET /api/template/blocks HTTP/1.1" 200 2603 "http://**.**.**.201:443/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "-"

2023/11/11 16:27:31 /home/runner/work/nginx-ui/nginx-ui/server/model/auth.go:56
[0.137ms] [rows:1] SELECT * FROM `auth_tokens` WHERE token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiaWNhb2xlaSIsImV4cCI6MTY5OTcxODk0OX0.-ONEes4SJ85InLzbWc75tusGPvmpXIsaiAZbx6oqaOA"
[GIN] 2023/11/11 - 16:27:31 | 200 |     568.476µs |   **.**.**.201 | GET      "/api/template/block/letsencrypt.conf"
**.**.**.201 - - [11/Nov/2023:16:27:31 +0800] "GET /api/template/block/letsencrypt.conf HTTP/1.1" 200 470 "http://**.**.**.201:443/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "-"

2023/11/11 16:27:31 /home/runner/work/nginx-ui/nginx-ui/server/model/auth.go:56
[0.128ms] [rows:1] SELECT * FROM `auth_tokens` WHERE token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiaWNhb2xlaSIsImV4cCI6MTY5OTcxODk0OX0.-ONEes4SJ85InLzbWc75tusGPvmpXIsaiAZbx6oqaOA"
[GIN] 2023/11/11 - 16:27:31 | 200 |      988.12µs |   **.**.**.201 | POST     "/api/ngx/build_config"
**.**.**.201 - - [11/Nov/2023:16:27:31 +0800] "POST /api/ngx/build_config HTTP/1.1" 200 1456 "http://**.**.**.201:443/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "-"

2023/11/11 16:27:31 /home/runner/work/nginx-ui/nginx-ui/server/model/auth.go:56
[0.122ms] [rows:1] SELECT * FROM `auth_tokens` WHERE token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiaWNhb2xlaSIsImV4cCI6MTY5OTcxODk0OX0.-ONEes4SJ85InLzbWc75tusGPvmpXIsaiAZbx6oqaOA"
2023/11/11 16:27:32 [notice] 14#14: signal 1 (SIGHUP) received from 108, reconfiguring
2023/11/11 16:27:32 [notice] 14#14: reconfiguring
2023/11/11 16:27:32 [notice] 14#14: using the "epoll" event method
2023/11/11 16:27:32 [notice] 14#14: start worker processes
2023/11/11 16:27:32 [notice] 14#14: start worker process 111
2023/11/11 16:27:32 [notice] 14#14: start worker process 112

2023/11/11 16:27:32 /home/runner/work/nginx-ui/nginx-ui/server/query/chat_gpt_logs.gen.go:315
[0.159ms] [rows:1] SELECT * FROM `chat_gpt_logs` WHERE `chat_gpt_logs`.`name` = "/etc/nginx/sites-available/abc.xyz.org" ORDER BY `chat_gpt_logs`.`name` LIMIT 1

2023/11/11 16:27:32 /home/runner/work/nginx-ui/nginx-ui/server/query/sites.gen.go:323
[0.199ms] [rows:0] SELECT * FROM `sites` WHERE `sites`.`path` = "/etc/nginx/sites-available/abc.xyz.org" AND `sites`.`deleted_at` IS NULL ORDER BY `sites`.`id` LIMIT 1

2023/11/11 16:27:32 /home/runner/work/nginx-ui/nginx-ui/server/model/cert.go:31 record not found
[0.156ms] [rows:0] SELECT * FROM `certs` WHERE `certs`.`filename` = "abc.xyz.org" AND `certs`.`deleted_at` IS NULL ORDER BY `certs`.`id` LIMIT 1
2023-11-11 16:27:32     WARN    api/domain.go:129       cert record not found
[GIN] 2023/11/11 - 16:27:32 | 200 |   114.66747ms |   **.**.**.201 | POST     "/api/domain/abc.xyz.org"
**.**.**.201 - - [11/Nov/2023:16:27:32 +0800] "POST /api/domain/abc.xyz.org HTTP/1.1" 200 5087 "http://**.**.**.201:443/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "-"
2023/11/11 16:27:32 [notice] 17#17: gracefully shutting down
2023/11/11 16:27:32 [notice] 16#16: gracefully shutting down
2023/11/11 16:27:32 [notice] 17#17: exiting
2023/11/11 16:27:32 [notice] 16#16: exiting
2023/11/11 16:27:32 [notice] 16#16: exit
2023/11/11 16:27:32 [notice] 17#17: exit
2023/11/11 16:27:32 [notice] 14#14: signal 17 (SIGCHLD) received from 17
2023/11/11 16:27:32 [notice] 14#14: worker process 17 exited with code 0
2023/11/11 16:27:32 [notice] 14#14: signal 29 (SIGIO) received
2023/11/11 16:27:32 [notice] 14#14: signal 17 (SIGCHLD) received from 16
2023/11/11 16:27:32 [notice] 14#14: worker process 16 exited with code 0
2023/11/11 16:27:32 [notice] 14#14: signal 29 (SIGIO) received
0xJacky commented 10 months ago

ws 的那个请求的消息可以看看吗，还是说就没有，看 Header 这个 http 请求没有 upgraded 啊

leic4u commented 10 months ago

ws 的那个请求的消息可以看看吗，还是说就没有，看 Header 这个 http 请求没有 upgraded 啊

是这个吗？是空的。 image

0xJacky commented 10 months ago

那就还是 websocket 的问题了

leic4u commented 10 months ago

那就还是 websocket 的问题了

这种怎么处理呢？🤣是我的配置问题吗？

leic4u commented 10 months ago

那就还是 websocket 的问题了

之前一直是使用的 ip:443 登录 nginx-ui 面板，我发现直接使用 ip:9000 / ip:80 / 域名:443 登录面板时可以正常颁发证书，就只有 ip:443 不行。

不过有点奇怪，昨天使用 ip:443 都还可以正常颁发证书，升级 beta-4 之后不行了。

hale177 commented 10 months ago

docker 是直接暴露的 80 和 443 端口到公网ip吗

阿里云ECS，有弹性公网 EIP ，就是直接暴露公网

hale177 commented 10 months ago

这个情况是 Nginx UI 自己的 WebSocket 无法访问，一般是配置问题

其实从 1.7.X 一直用到现在1.9.x ，基本证书全靠 手动触发的更新（http-01），定时自动轮转更新机制感觉并没有运转（只在启停的时候会看到证书更新的log，其他时候没见过），好多都是被发现过期了，提醒上去手动更新 。

0xJacky commented 10 months ago

等 gocron v2.0.0 发正式版了我重构一次这个部分

0xJacky commented 10 months ago

这次应该修好了，是我之前没做好错误处理导致 channel 造成阻塞，这项更新将会在 v2.beta.5 中发布