search

NginxProxyManager / nginx-proxy-manager

Docker container for managing Nginx proxy hosts with a simple, powerful interface
https://nginxproxymanager.com
MIT License
23.43k stars 2.72k forks source link

Complete crash when requesting a second wild-card cert from GoDaddy with DNS #1628

Open JohnGalt1717 opened 2 years ago

JohnGalt1717 commented 2 years ago

Checklist

Describe the bug If you try and add a second wild card cert from the SSL tab using go-daddy (not sure if it does this with others) you'll get an internal error about an npm folder in /letsencrypt/live not existing. Anything else you try and do in the session will error although the existing proxies will continue to function. If you restart the container, it will crash on boot. The only way to work around is to copy one of the other npm folders into the one it's looking for in the log and then it will start.

Nginx Proxy Manager Version 2.9.7

To Reproduce Steps to reproduce the behavior:

  1. Go to SSL
  2. Click on Add SSL Cert
  3. Add wildcard (*.example.com) and choose go daddy and fill in secret and key click create.
  4. Add a second with the same information for a different domain (i.e. *.example2.com), click create => error as described.

Expected behavior Should add the second certificate without error and not bork nginx manager entirely.

Operating System Debian Linux

Additional context ❯ /data/nginx/redirection_host/1.conf nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/npm-8/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/npm-8/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

chaptergy commented 2 years ago

Could you please provide the letsencrypt logs (see https://github.com/jc21/nginx-proxy-manager/issues/1271#user-content-certificate-error)

JohnGalt1717 commented 2 years ago

I don't think it's in there (because I "fixed" it by copying the directory in) but the one that failed to create the directory etc was npm-8:

2021-12-03 09:39:48,732:DEBUG:certbot._internal.main:certbot version: 1.17.0
2021-12-03 09:39:48,735:DEBUG:certbot._internal.main:Location of certbot entry point: /opt/certbot/bin/certbot
2021-12-03 09:39:48,735:DEBUG:certbot._internal.main:Arguments: ['--non-interactive', '--quiet', '--config', '/etc/letsencrypt.ini', '--preferred-challenges', 'dns,http', '--disable-hook-validation']
2021-12-03 09:39:48,735:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#certbot-dns-godaddy:dns-godaddy,PluginEntryPoint#dns-godaddy,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2021-12-03 09:39:48,884:DEBUG:certbot._internal.log:Root logging level set at 40
2021-12-03 09:39:48,887:DEBUG:certbot.display.util:Notifying user: Processing /etc/letsencrypt/renewal/npm-11.conf
2021-12-03 09:39:49,014:DEBUG:certbot._internal.plugins.selection:Requested authenticator <certbot._internal.cli.cli_utils._Default object at 0x7fb4eeaac8> and installer <certbot._internal.cli.cli_utils._Default object at 0x7fb4eeaac8>
2021-12-03 09:39:49,014:DEBUG:certbot._internal.cli:Var pref_challs=dns,http (set by user).
2021-12-03 09:39:49,014:DEBUG:certbot._internal.cli:Var key_type=ecdsa (set by user).
2021-12-03 09:39:49,015:DEBUG:certbot._internal.cli:Var elliptic_curve=secp384r1 (set by user).
2021-12-03 09:39:49,015:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2021-12-03 09:39:49,015:DEBUG:certbot._internal.cli:Var webroot_map={'webroot_path'} (set by user).
2021-12-03 09:39:49,015:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2021-12-03 09:39:49,120:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): r3.o.lencr.org:80
2021-12-03 09:39:49,316:DEBUG:urllib3.connectionpool:http://r3.o.lencr.org:80 "POST / HTTP/1.1" 200 503
2021-12-03 09:39:49,318:DEBUG:certbot.ocsp:OCSP response for certificate /etc/letsencrypt/archive/npm-11/cert5.pem is signed by the certificate's issuer.
2021-12-03 09:39:49,326:DEBUG:certbot.ocsp:OCSP certificate status for /etc/letsencrypt/archive/npm-11/cert5.pem is: OCSPCertStatus.GOOD
2021-12-03 09:39:49,341:DEBUG:certbot.display.util:Notifying user: Certificate not yet due for renewal
2021-12-03 09:39:49,344:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2021-12-03 09:39:49,344:DEBUG:certbot.display.util:Notifying user: Processing /etc/letsencrypt/renewal/npm-2.conf
2021-12-03 09:39:49,350:DEBUG:certbot._internal.cli:Var pref_challs=dns,http (set by user).
2021-12-03 09:39:49,351:DEBUG:certbot._internal.cli:Var key_type=ecdsa (set by user).
2021-12-03 09:39:49,351:DEBUG:certbot._internal.cli:Var elliptic_curve=secp384r1 (set by user).
2021-12-03 09:39:49,351:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2021-12-03 09:39:49,352:DEBUG:certbot._internal.cli:Var webroot_map={'webroot_path'} (set by user).
2021-12-03 09:39:49,352:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2021-12-03 09:39:49,478:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): r3.o.lencr.org:80
2021-12-03 09:39:49,630:DEBUG:urllib3.connectionpool:http://r3.o.lencr.org:80 "POST / HTTP/1.1" 200 503
2021-12-03 09:39:49,633:DEBUG:certbot.ocsp:OCSP response for certificate /etc/letsencrypt/archive/npm-2/cert5.pem is signed by the certificate's issuer.
2021-12-03 09:39:49,635:DEBUG:certbot.ocsp:OCSP certificate status for /etc/letsencrypt/archive/npm-2/cert5.pem is: OCSPCertStatus.GOOD
2021-12-03 09:39:49,638:DEBUG:certbot.display.util:Notifying user: Certificate not yet due for renewal
2021-12-03 09:39:49,640:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2021-12-03 09:39:49,641:DEBUG:certbot.display.util:Notifying user: Processing /etc/letsencrypt/renewal/npm-20.conf
2021-12-03 09:39:49,769:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): r3.o.lencr.org:80
2021-12-03 09:39:49,865:DEBUG:urllib3.connectionpool:http://r3.o.lencr.org:80 "POST / HTTP/1.1" 200 503
2021-12-03 09:39:49,868:DEBUG:certbot.ocsp:OCSP response for certificate /etc/letsencrypt/archive/npm-20/cert1.pem is signed by the certificate's issuer.
2021-12-03 09:39:49,869:DEBUG:certbot.ocsp:OCSP certificate status for /etc/letsencrypt/archive/npm-20/cert1.pem is: OCSPCertStatus.GOOD
2021-12-03 09:39:49,871:DEBUG:certbot.display.util:Notifying user: Certificate not yet due for renewal
2021-12-03 09:39:49,873:DEBUG:certbot._internal.plugins.selection:Requested authenticator dns-godaddy and installer None
2021-12-03 09:39:49,873:DEBUG:certbot.display.util:Notifying user: Processing /etc/letsencrypt/renewal/npm-3.conf
2021-12-03 09:39:49,877:DEBUG:certbot._internal.cli:Var pref_challs=dns,http (set by user).
2021-12-03 09:39:49,878:DEBUG:certbot._internal.cli:Var key_type=ecdsa (set by user).
2021-12-03 09:39:49,878:DEBUG:certbot._internal.cli:Var elliptic_curve=secp384r1 (set by user).
2021-12-03 09:39:49,878:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2021-12-03 09:39:49,878:DEBUG:certbot._internal.cli:Var webroot_map={'webroot_path'} (set by user).
2021-12-03 09:39:49,878:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2021-12-03 09:39:49,997:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): r3.o.lencr.org:80
2021-12-03 09:39:50,092:DEBUG:urllib3.connectionpool:http://r3.o.lencr.org:80 "POST / HTTP/1.1" 200 503
2021-12-03 09:39:50,094:DEBUG:certbot.ocsp:OCSP response for certificate /etc/letsencrypt/archive/npm-3/cert5.pem is signed by the certificate's issuer.
2021-12-03 09:39:50,096:DEBUG:certbot.ocsp:OCSP certificate status for /etc/letsencrypt/archive/npm-3/cert5.pem is: OCSPCertStatus.GOOD
2021-12-03 09:39:50,098:DEBUG:certbot.display.util:Notifying user: Certificate not yet due for renewal
2021-12-03 09:39:50,101:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2021-12-03 09:39:50,102:DEBUG:certbot.display.util:Notifying user: Processing /etc/letsencrypt/renewal/npm-4.conf
2021-12-03 09:39:50,106:DEBUG:certbot._internal.cli:Var pref_challs=dns,http (set by user).
2021-12-03 09:39:50,106:DEBUG:certbot._internal.cli:Var key_type=ecdsa (set by user).
2021-12-03 09:39:50,106:DEBUG:certbot._internal.cli:Var elliptic_curve=secp384r1 (set by user).
2021-12-03 09:39:50,106:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2021-12-03 09:39:50,107:DEBUG:certbot._internal.cli:Var webroot_map={'webroot_path'} (set by user).
2021-12-03 09:39:50,107:DEBUG:certbot._internal.cli:Var webroot_path=/data/letsencrypt-acme-challenge (set by user).
2021-12-03 09:39:50,193:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): r3.o.lencr.org:80
2021-12-03 09:39:50,267:DEBUG:urllib3.connectionpool:http://r3.o.lencr.org:80 "POST / HTTP/1.1" 200 503
2021-12-03 09:39:50,270:DEBUG:certbot.ocsp:OCSP response for certificate /etc/letsencrypt/archive/npm-4/cert5.pem is signed by the certificate's issuer.
2021-12-03 09:39:50,271:DEBUG:certbot.ocsp:OCSP certificate status for /etc/letsencrypt/archive/npm-4/cert5.pem is: OCSPCertStatus.GOOD
2021-12-03 09:39:50,273:DEBUG:certbot.display.util:Notifying user: Certificate not yet due for renewal
2021-12-03 09:39:50,275:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2021-12-03 09:39:50,276:DEBUG:certbot.display.util:Notifying user:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2021-12-03 09:39:50,276:DEBUG:certbot.display.util:Notifying user: The following certificates are not due for renewal yet:
2021-12-03 09:39:50,276:DEBUG:certbot.display.util:Notifying user:   /etc/letsencrypt/live/npm-11/fullchain.pem expires on 2022-03-01 (skipped)
  /etc/letsencrypt/live/npm-2/fullchain.pem expires on 2022-03-01 (skipped)
  /etc/letsencrypt/live/npm-20/fullchain.pem expires on 2022-03-03 (skipped)
  /etc/letsencrypt/live/npm-3/fullchain.pem expires on 2022-03-01 (skipped)
  /etc/letsencrypt/live/npm-4/fullchain.pem expires on 2022-03-01 (skipped)
2021-12-03 09:39:50,276:DEBUG:certbot.display.util:Notifying user: No renewals were attempted.
2021-12-03 09:39:50,276:DEBUG:certbot.display.util:Notifying user: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2021-12-03 09:39:50,277:DEBUG:certbot._internal.renewal:no renewal failures
chaptergy commented 2 years ago

Yeah, as you said there is nothing in there, since this is most likely not the log of when npm-8 was created. If you can replicate the issue please provide the logs of what happens when the error occurs.

JohnGalt1717 commented 2 years ago

One would assume that it's a pretty straight forward repro though. Sorry I didn't get the logs, but it's a live system with external dependencies so after I reproduced the issue once I fixed it and put it back in production to minimize downtime.

chaptergy commented 2 years ago

It is much easier when you actually have a GoDaddy domain :P
Just let me know if you ever run into this issue again and have logs to help debug it.

JohnGalt1717 commented 2 years ago

... almost certain it will happen with any DNS verification if you just do 2 separate wildcards....

chaptergy commented 2 years ago

Well, I am not able to reproduce it with other providers, requesting multiple wildcard certificates for the same domain, e.g. *.example.com works as expected, and restarting npm does not cause it to crash on boot.

the1ts commented 2 years ago

Yes, can confirm that Hetzner can have multiple wildcard certs for different domains without issues. Checked and the new cert is created fine.

ch4ox commented 2 years ago

I once had a similar problem where something like this happened and I had to fix paths in the database manually.

I think the steps I took were 1. creating a wildcard cert (Hetzner), 2. attaching this cert to a host, 3. deleting the cert without updating the host afterwards. A restart finally killed it for good.

Maybe something like that happened here as well?

tree-white commented 2 years ago

When I applied for a wildcard certificate, there was an error error, and after I tried to restart, there was a nginx: [emerg] cannot load certificate.

❯ Enabling IPV6 in hosts: /etc/nginx/conf.d
  ❯ /etc/nginx/conf.d/production.conf
  ❯ /etc/nginx/conf.d/default.conf
  ❯ /etc/nginx/conf.d/include/block-exploits.conf
  ❯ /etc/nginx/conf.d/include/force-ssl.conf
  ❯ /etc/nginx/conf.d/include/letsencrypt-acme-challenge.conf
  ❯ /etc/nginx/conf.d/include/assets.conf
  ❯ /etc/nginx/conf.d/include/ssl-ciphers.conf
  ❯ /etc/nginx/conf.d/include/proxy.conf
  ❯ /etc/nginx/conf.d/include/ip_ranges.conf
  ❯ /etc/nginx/conf.d/include/resolvers.conf
❯ Enabling IPV6 in hosts: /data/nginx
  ❯ /data/nginx/dead_host/1.conf
  ❯ /data/nginx/default_host/site.conf
  ❯ /data/nginx/proxy_host/14.conf
  ❯ /data/nginx/proxy_host/7.conf
  ❯ /data/nginx/proxy_host/13.conf
  ❯ /data/nginx/proxy_host/2.conf
  ❯ /data/nginx/proxy_host/9.conf
  ❯ /data/nginx/proxy_host/4.conf
  ❯ /data/nginx/proxy_host/11.conf
  ❯ /data/nginx/proxy_host/8.conf
  ❯ /data/nginx/proxy_host/5.conf
  ❯ /data/nginx/proxy_host/10.conf
  ❯ /data/nginx/proxy_host/12.conf
nginx: [emerg] cannot load certificate "/etc/docker/letsencrypt/live/npm-19/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/docker/letsencrypt/live/npm-19/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
spuxx1701 commented 2 years ago

I'm running into the same issue. Did you find a solution @tree-white?

spuxx1701 commented 2 years ago

The issue was caused by a proxy_host that was assigned an ssl certificate that had been deleted. I managed to fix it by navigating to the data volume and into /nginx/proxy_host, and deleting the *.conf files that were referring to the deleted certificate.

tree-white commented 2 years ago

I'm running into the same issue. Did you find a solution @tree-white?

I forgot after a long time, but in the end I remembered redeployed and only applied for a wildcard certificate.

github-actions[bot] commented 9 months ago

Issue is now considered stale. If you want to keep it open, please comment :+1:

PlamenGeorgievKostadinov commented 3 weeks ago

same problem here