search

jlesage / docker-nginx-proxy-manager

Docker container for Nginx Proxy Manager
MIT License
756 stars 167 forks source link

Trapped in a crash loop after deleting broken expired certs #246

Open themaster567 opened 2 years ago

themaster567 commented 2 years ago

SOLVED, SEE EDIT 3

I was having some problems connecting to one of my domains with ssl that I had set up, so I went into the logs and discovered that it was endlessly failing to renew a whole ton of certificates. I also couldn't connect NGINX Proxy Manager's web portal at all due to more strict SSL settings I had set. I then went into NGINX Proxy Manager through IP and looked at the SSL page and discovered that nearly every single cert had been expired for months. Trying to renew any of them manually just threw an internal service error. I eventually solved the initial problem by discovering that the certs would work if I deleted them and then remade them.

This is where I made my fatal mistake. I went through and systematically deleted all the brokens certs all at once. I then started one by one creating new certs for them, but then it started throwing internal service errors again. I restarted the container, and now it's caught in a boot loop of trying to load a cert that doesn't exist, failing to do so, and restarting.

❯ Enabling IPV6 in hosts: /etc/nginx/conf.d,
  ❯ /etc/nginx/conf.d/production.conf,
  ❯ /etc/nginx/conf.d/default.conf,
  ❯ /etc/nginx/conf.d/include/force-ssl.conf,
  ❯ /etc/nginx/conf.d/include/ip_ranges.conf,
  ❯ /etc/nginx/conf.d/include/assets.conf,
  ❯ /etc/nginx/conf.d/include/block-exploits.conf,
  ❯ /etc/nginx/conf.d/include/letsencrypt-acme-challenge.conf,
  ❯ /etc/nginx/conf.d/include/ssl-ciphers.conf,
  ❯ /etc/nginx/conf.d/include/proxy.conf,
  ❯ /etc/nginx/conf.d/include/resolvers.conf,
❯ Enabling IPV6 in hosts: /data/nginx,
  ❯ /data/nginx/proxy_host/57.conf,
  ❯ /data/nginx/redirection_host/1.conf,
  ❯ /data/nginx/proxy_host/5.conf,
  ❯ /data/nginx/proxy_host/31.conf,
  ❯ /data/nginx/proxy_host/62.conf,
  ❯ /data/nginx/proxy_host/38.conf,
  ❯ /data/nginx/proxy_host/59.conf,
  ❯ /data/nginx/proxy_host/11.conf,
  ❯ /data/nginx/proxy_host/12.conf,
  ❯ /data/nginx/proxy_host/40.conf,
  ❯ /data/nginx/proxy_host/9.conf,
  ❯ /data/nginx/proxy_host/48.conf,
  ❯ /data/nginx/proxy_host/36.conf,
  ❯ /data/nginx/proxy_host/44.conf,
  ❯ /data/nginx/proxy_host/35.conf,
  ❯ /data/nginx/proxy_host/51.conf,
  ❯ /data/nginx/proxy_host/55.conf,
  ❯ /data/nginx/proxy_host/4.conf,
  ❯ /data/nginx/proxy_host/18.conf,
  ❯ /data/nginx/proxy_host/52.conf,
  ❯ /data/nginx/proxy_host/13.conf,
  ❯ /data/nginx/proxy_host/56.conf,
  ❯ /data/nginx/proxy_host/50.conf,
  ❯ /data/nginx/proxy_host/17.conf,
  ❯ /data/nginx/proxy_host/53.conf,
  ❯ /data/nginx/proxy_host/23.conf,
  ❯ /data/nginx/proxy_host/33.conf,
  ❯ /data/nginx/proxy_host/10.conf,
  ❯ /data/nginx/proxy_host/16.conf,
  ❯ /data/nginx/proxy_host/60.conf,
  ❯ /data/nginx/proxy_host/61.conf,
  ❯ /data/nginx/proxy_host/39.conf,
  ❯ /data/nginx/proxy_host/8.conf,
  ❯ /data/nginx/proxy_host/28.conf,
  ❯ /data/nginx/proxy_host/49.conf,
  ❯ /data/nginx/proxy_host/30.conf,
  ❯ /data/nginx/proxy_host/46.conf,
  ❯ /data/nginx/proxy_host/7.conf,
  ❯ /data/nginx/default_host/site.conf,
nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/npm-15/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/npm-15/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

It outputs this forever. I can't log into the web portal because it's never loaded for more than a few seconds. I'm not sure what to do.

EDIT: I now recall fixing this before, since I think I actually managed to have this problem on a smaller scale awhile back, but it involved connecting to a command line inside of the container, copying existing existing unrelated cert folders to the location of the the missing ones and then renaming them to what it was looking for. The problem is that I would have to do this about 30+ times. If there's any way to avoid that at all, that would be great.

EDIT 2: I think what could fix it would be if I were able to set every web page to not use SSL, then I don't think it would be trying to load them. I'm just not sure how to do that without access to the web interface.

EDIT 3: I fixed it doing the long and tedious way I mentioned in edit 1. I'm leaving this ticket open though just in case anyone has any ideas on how this could've been done faster, as well as a word of warning for not doing something so stupid in the first place.

ThijmenGThN commented 1 year ago

Solved this myself, following what you've mentioned.

The error: image

What I did to solve it:

  1. Exec into the container. docker exec -it <container_name> sh

  2. Navigate to the correct dir. cd /etc/letsencrypt/live

  3. Copied an existing folder and named it as the missing one. cp -r npm-14 npm-15 image