datamade / how-to

📚 Doing all sorts of things, the DataMade way
MIT License
84 stars 12 forks source link

Nginx Certbot installer spawns rogue process, causing cert renewals to fail #160

Closed hancush closed 9 months ago

hancush commented 3 years ago

Description

Certbot renewals were running but failing on the SSCE staging server. I used the --dry-run option to test these failures.

sudo certbot renew --dry-run

First, I needed to install python3-acme to issue correctly formed requests, per this thread.

Then, I needed to tell Certbot not to use the Nginx installer, as commands fail with messages about failing to restart Nginx. This is a well-known issue having to do with Certbot spawing Nginx outside of the usual service daemon, preventing the desired process from using ports 80 and 443: https://github.com/certbot/certbot/issues/5486. I achieved this by updating the Certbot config file at /etc/letsencrypt/cli.ini:

# Because we are using logrotate for greater flexibility, disable the
# internal certbot logrotation.
max-log-backups = 0

[renewalparams]
installer = none
pre-hook = service nginx stop
post-hook = service nginx start

I'm not sure why this doesn't seem to be affecting all of our servers, but I wanted to capture these steps someplace, in case it continues to be an issue.

hancush commented 3 years ago

SSCE site was down. I checked that the supervisor process was running:

ubuntu@ip-10-0-0-75:~$ sudo supervisorctl status
ssce                             RUNNING   pid 3051, uptime 155 days, 3:15:51

Then I checked on Nginx:

ubuntu@ip-10-0-0-75:~$ sudo service nginx status
...
Jan 05 19:35:45 ip-10-0-0-75 systemd[1]: Failed to start nginx - high performance web serve

Restarting Nginx brought the app back online:

ubuntu@ip-10-0-0-75:~$ sudo service nginx restart
ubuntu@ip-10-0-0-75:~$ sudo service nginx status
...
Jan 19 21:39:23 ip-10-0-0-75 systemd[1]: Started nginx - high performance web server.
smcalilly commented 9 months ago

SSCE has been migrated to Heroku, closing this.