Certificate renewal failed

[jamesgardnergeovation]

In this case it was because the because letsencrypt directory mounted wasn't actually the correct one

CAUTION: Think this wouldn't happen, even on auto-renew because it needs to restart itself.

The traceback from the gateway-lite logs is:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/certbot/renewal.py", line 63, in _reconstitute
    renewal_candidate = storage.RenewableCert(full_path, config)
  File "/usr/lib/python2.7/site-packages/certbot/storage.py", line 434, in __init__
    self._check_symlinks()
  File "/usr/lib/python2.7/site-packages/certbot/storage.py", line 493, in _check_symlinks
    "expected {0} to be a symlink".format(link))
CertStorageError: expected /etc/letsencrypt/live/www.runfriendlyexperiments.com/cert.pem to be a symlink
Renewal configuration file /etc/letsencrypt/renewal/www.runfriendlyexperiments.com.conf is broken. Skipping.

Here's how to handle a manual renewal:

$ docker ps
CONTAINER ID        IMAGE                                         COMMAND                  CREATED             STATUS              PORTS
                                    NAMES
7f40aa3e63f4        thejimmyg/gateway-lite:0.2.4                  "node bin/gateway-li…"   2 months ago        Up 17 seconds       0.0.0.0:80->80/tcp, 3000/tcp, 0.0.0.0:443->443/tcp, 8000/tcp   run-friendly-experiment-1_gateway_1
...
$ docker logs 7f40aa3e63f4 &> logs
$ vim logs                                                   
$ docker exec -it 7f40aa3e63f4 /bin/sh
/app # ps aux 
PID   USER     TIME  COMMAND
    1 root      0:00 node bin/gateway-lite.js --https-port 443 --port 80 --domain domain --cert domain/www.runfriendlyexperiments.c
   16 root      0:00 /bin/sh
   22 root      0:00 ps aux
/app # certbot -q renew
/app # exit

$ sudo cp letsencrypt/live/www.runfriendlyexperiments.com/fullchain.pem domain/www.runfriendlyexperiments.com/sni/cert.pem
$ sudo cp letsencrypt/live/www.runfriendlyexperiments.com/privkey.pem domain/www.runfriendlyexperiments.com/sni/key.pem
$ docker restart  7f40aa3e63f4

[thejimmyg]

I've created an installCertificates() function which takes the latest certificate from /etc/letsencrypt for each domain and installs them into the domain structure.

Pull request is here:

https://github.com/thejimmyg/gateway-lite/pull/5

I've pushed this as thejimmyg/gateway-lite:test to docker hub for testing.

[thejimmyg]

Here's how you can test renewal once you already have a renewed certificate.

# cd letsencrypt/live/docker.jimmyg.org
# ls -lah
total 12K
drwxr-xr-x 2 root root 4.0K Feb 14 10:37 .
drwx------ 5 root root 4.0K Mar 22 14:18 ..
-rw-r--r-- 1 root root  682 Dec 16 10:46 README
lrwxrwxrwx 1 root root   41 Feb 14 10:37 cert.pem -> ../../archive/docker.jimmyg.org/cert2.pem
lrwxrwxrwx 1 root root   42 Feb 14 10:37 chain.pem -> ../../archive/docker.jimmyg.org/chain2.pem
lrwxrwxrwx 1 root root   46 Feb 14 10:37 fullchain.pem -> ../../archive/docker.jimmyg.org/fullchain2.pem
lrwxrwxrwx 1 root root   44 Feb 14 10:37 privkey.pem -> ../../archive/docker.jimmyg.org/privkey2.pem
# rm cert.pem 
# ln -s ../../archive/docker.jimmyg.org/cert1.pem cert.pem
# rm chain.pem 
# ln -s ../../archive/docker.jimmyg.org/chain1.pem chain.pem
# rm fullchain.pem 
# ln -s ../../archive/docker.jimmyg.org/fullchain1.pem fullchain.pem
# rm privkey.pem 
# ln -s ../../archive/docker.jimmyg.org/privkey1.pem privkey.pem
# ls -la
total 12
drwxr-xr-x 2 root root 4096 Mar 22 20:41 .
drwx------ 5 root root 4096 Mar 22 14:18 ..
-rw-r--r-- 1 root root  682 Dec 16 10:46 README
lrwxrwxrwx 1 root root   41 Mar 22 20:41 cert.pem -> ../../archive/docker.jimmyg.org/cert1.pem
lrwxrwxrwx 1 root root   42 Mar 22 20:41 chain.pem -> ../../archive/docker.jimmyg.org/chain1.pem
lrwxrwxrwx 1 root root   46 Mar 22 20:41 fullchain.pem -> ../../archive/docker.jimmyg.org/fullchain1.pem
lrwxrwxrwx 1 root root   44 Mar 22 20:41 privkey.pem -> ../../archive/docker.jimmyg.org/privkey1.pem
# cd ../../archive/docker.jimmyg.org/
# ls
cert1.pem  cert2.pem  chain1.pem  chain2.pem  fullchain1.pem  fullchain2.pem  privkey1.pem  privkey2.pem
# mkdir latest
# mv *2.pem latest/
# ls

Now remove the latest certificate:

cd ~
rm domain/docker.jimmyg.org/sni/*.pem

Now restart the server so it automatically installs the old certificate:

time docker-compose pull && time docker-compose down && time docker-compose up -d && docker-compose logs --tail="all" -f

Wait 24 hours and a renewal should have occurred.

[thejimmyg]

Hopefully this is resolved in 0.2.16 with #5. Marking as closed, we can re-open if the issue continues.