Closed sylvainar closed 6 years ago
Solved it !
- name: Run certbot generation for each host
command: /opt/certbot/certbot-auto --apache -n -d {{ item['host'] }} --email {{ item['admin_email'] }} --redirect --agree-tos
with_items: "{{ vhost_sites }}"
I'm gonna make a PR if I have the time to do it properly.
@sylvainar - Thanks for the example!
I think I will try to incorporate this in as modular a way as possible, since I'm currently using the role on both Apache and Nginx-based servers (haven't tried Caddy, though it does it's own thing with Let's Encrypt).
The task above could work, but I think I'd want to use a custom variable to drive what hosts get certs, because on many of my servers, I have multiple hosts, some of which should get certs, some of which shouldn't.
But I definitely want to add this functionality.
Great, I'm staying tuned!
How about using the webroot plugin instead.
certbot_hosts:
- webroot: "/var/www/drupal/drupal"
email: "m@oxy.fi"
domain: "drupal.dev www.drupal.dev"
- name: Obtain certificates.
command: "{{ certbot_script }} certonly --webroot -w {{ item.webroot }} --email {{ item.email }} --agree-tos --keep --expand -d {{ item.domain.split()|join('-d ') }}"
with_items: certbot_hosts
This will create the ceritificate in/etc/letsencrypt/live/drupal.dev/cert.pem
(Ubuntu 16.04) and then the user configures the webserver to point to it separately.
We should probably define a certbot_certificate_dir
that points to the live/
subdir on all different distros. This way you just pass that on the the nginx/ansible role in your playbook.
@oxyc that would work for me; I don't like Certbot touching my configs anyways, I just want it to automatically generate a cert then let me know where it is so I can update my nginx/apache config.
Automatically retrieving certificates on first run (i.e. before the cron job takes effect) would be extremely useful! The sketch from @oxyc mentioned above looks nice. I guess a few things need to be considered.
/etc/letsencrypt/live/
) that the web server can use to start properly and make the webroot plugin work. Personally, I would consider it inconvenient and hacky to have the web server configured with plain http only or with a different certificate path in the first run, then having certbot retrieve a valid certificate and then some other Ansible role that sets up the web server properly. Maybe we can come up with a better solution. Possibly this role could detect if this is the very first run (no cert present) and use the certbot standalone plugin instead. Alternatively, it could create a snakeoil/selfsigned certificate and put it in the right place.Beside that, +1 for only retrieving certificates, not touching web server's configuration.
I like this idea, some way to automatically manage this. I'm using htis with @geerlingguy 's Apache config too. I ran it by hand the first time to test it out. But found this ticket in hopes to find an automated way.
Another thought, you can't request too many certs too often. Maybe we add some "lock file" that we check the timestamp of creation to determine if we should re-run it? (Like a once only type thing?)
Probably it's reasonable to check for the existence and validity of the cert and only obtain it when necessary. Generally this shouldn't happen too often since one should have a cron job for this. But iirc the certbot docs say that on renew it won't do anything if the cert is still valid for a certain time frame. Doesn't it do the same for non-renew certonly retrieval. Don't know if the rate limiting is an issue. We should take care of this.
Another friend just pinged me and said it could be as simple as this (if you have the vhosts set up otherwise and responding to requests for the non-SSL version):
# Variable:
certbot_vhosts:
- email: johndoe@example.com
domains:
- example.com
- www.example.com
# In role's main.yml.
- include: certs.yml
with_items: certbot_vhosts
# In certs.yml
- name: Check if certificate already exists.
stat:
path: /etc/letsencrypt/live/{{ item.domains | first }}/cert.pem
register: letsencrypt_cert
- name: Generate new certificate if one doesn't exist.
shell: certbot certonly --apache --agree-tos -n -m {{ item.email }} -d {{ item.domains | join(',') }}
when: not letsencrypt_cert.stat.exists
(pseudocode, don't sue me if it doesn't work... and he's using it only with Apache).
Well that's more or less what I proposed on the second post :wink:
Personally, I don't like Certbot changing my webserver config. If you agree to this and want the role to behave according to this, the --apache
flag might be inappropriate.
In my opinion, the standalone
authenticator would fit the needs best because it does not require a fully configured webserver that can't be working anyway at this point, because there is no TLS certificate yet. I think we would need a setting that allows the role to know how to stop a potentially running webserver (to free ports 80 and 443). I.e. a setting like certbot_before_initial_retrieval = systemctl stop nxinx
that is executed when the letsencrypt_cert.stat.exists
check mentioned above fails. This would also work well if there is an already running webserver for domain x and now Ansible-Certbot is told to obtain a cert for domain y that is not yet configured in the webserver.
On the other hand, the renew cronjob should use the webroot
authenticator to avoid unnecessary downtimes. Docs on renew
say:
The same plugin and options that were used at the time the certificate was originally issued will be used for the renewal attempt, unless you specify other plugins or options.
So when webroot
should be used for renewal instead of standalone
, we would need to specify this explicitly for the cron job. But that should be easy.
I would like to work on a draft PR for this, but I'm a bit busy right now, so I don't know when I can come up with this :/
EDIT: To make the workflow that I have in mind clear: In a playbook, one lists the certbot role before the webserver role. This way, certbot can retrieve certificates for new domains and then, when the webserver config is applied, the certificate that is referred in the vhost is already there and the webserver successfully starts up.
How would you implement this workflow for a migration server that does not have the DNS pointing to it yet? it seems like its not possible as letsencrypt cant validate that you own the domain?
seems like i would have to point dns first, then run a second playbook to run letsencrypt and install them to the right place
and if you have multiple web servers you only want to generate the certs on one, then propogate them to the rest...
This is indeed a problem. It seems it is not possible to obtain a LE cert w/o a valid DNS entry (when going without some dirty hacks). Maybe we cannot do more in such a situation. Guess I would make the certbot role conditional, depending on some host variable that determines if this is a staging or production server. But then one can get in trouble with the webserver config.. I see what you mean..
Maybe we can come up with a partial solution eventually, but at first it would be nice to have the basic setup working.
Yeah, I'm 99% sure it isn't possible (or at least not easy without some really hacky workarounds) to generate certs on something like a dev server or local environment without any DNS available through the general public Internet.
I'm currently using this on a server.
--webroot
to obtain certificates and service apache2 reload
to re-read the certificate.certbot_webserver_port: 80
certbot_renew_hook: "service apache2 reload"
certbot_certificates:
- email: "m@oxy.fi"
webroot: "{{ packagist_deploy_docroot }}"
domains:
- "packagist.{{ drupal_domain }}"
- email: "m@oxy.fi"
webroot: "{{ drupal_core_path }}"
domains:
- "{{ drupal_domain }}"
- "www.{{ drupal_domain }}"
---
- name: Detect if webserver is running.
command: "lsof -i :{{ certbot_webserver_port }}"
register: certbot_webserver_running
failed_when: false
changed_when: false
- name: Check if certificates already exists.
stat:
path: "/etc/letsencrypt/live/{{ item.domains|first }}/cert.pem"
register: certbot_certificate_paths
with_items: "{{ certbot_certificates }}"
- name: Generate certificates (standalone).
command: "{{ certbot_script }} certonly --standalone --email {{ item.1.email }} -n --agree-tos --keep -d {{ item.1.domains|join(',') }}"
when:
- not certbot_certificate_paths.results[item.0].stat.exists
- certbot_webserver_running.rc == 1
with_indexed_items: "{{ certbot_certificates }}"
- name: Generate certificates (webserver running)
command: "{{ certbot_script }} certonly --webroot -w {{ item.1.webroot }} --email {{ item.1.email }} -n --agree-tos --keep -d {{ item.1.domains|join(',') }} --post-hook '{{ certbot_renew_hook }}'"
when:
- not certbot_certificate_paths.results[item.0].stat.exists
- certbot_webserver_running.rc == 0
with_indexed_items: "{{ certbot_certificates }}"
- name: Update certificate domains.
command: "{{ certbot_script }} certonly --cert-name {{ item.1.domains|first }} --webroot -w {{ item.1.webroot }} -d {{ item.1.domains|join(',') }} -n --post-hook '{{ certbot_renew_hook }}'"
when:
- certbot_certificate_paths.results[item.0].stat.exists
- certbot_webserver_running.rc == 0
with_indexed_items: "{{ certbot_certificates }}"
register: certbot_certificate_update
changed_when: "'Your certificate and chain have been saved' in certbot_certificate_update.stdout"
- name: Add cron job for certbot renewal (if configured).
cron:
name: "Certbot automatic renewal of {{ item.domains|first }}"
job: "{{ certbot_script }} renew --cert-name {{ item.domains|first }} --webroot -w {{ item.webroot }} --post-hook '{{ certbot_renew_hook }}' -n --quiet --no-self-upgrade"
minute: "{{ certbot_auto_renew_minute }}"
hour: "{{ certbot_auto_renew_hour }}"
user: "{{ certbot_auto_renew_user }}"
when: certbot_auto_renew
with_items: "{{ certbot_certificates }}"
Several functionalities required Cerbot v0.10.0 and it was just too much trouble to make it work for older versions (the ones installed using package managers). --post-hook
(for certonly), --cert-name
and --renew-with-new-domains
weren't available.
Also note that --post-hook
wont run unless the certificate is regenerated, which is why I call it certbot_renew_hook
.
Edit: I can also note that the apache plugin couldn't figure out my vhost files so webroot or standalone + apache shutdown were my only options.
Thanks for the great reference.
My requirement is:
I took this approach here: https://github.com/ScalaWilliam/git-work/commit/916552c4fb5ae963782faf0114da148f81d092db
In the setup step, nginx configuration includes an empty 'https configuration' file. But in the ssl step, it sets that file's contents and then follows to reload the service.
I'm not too great with Ansible though but sharing my findings in case anyone finds it useful :-)
Today I built a server where I had basic auth to access the host, therefore relying on the webroot plugin doesn't really cover all scenarios.
I'm still toying with this. The chicken-and-egg problem is the most annoying—I need to manage Nginx/Apache/whatever's configuration, and I need to tell it there's a cert somewhere.
But the cert is not there until Let's Encrypt / Certbot generates the cert. But I can't always guarantee Let's Encrypt / Certbot will be installed and generate the cert before Nginx is installed / running. So besides an awkward dance of changing Nginx configs per server (or Apache per virtualhost) every time there's a new cert for a new domain (renewals are a different beast, but much easier), I'm using self-signed certs of snakeoil certs with the OS at first, then having certbot replace them. This way the server can start up initially using a cert in the LE path, then after LE creates the new real cert, we restart and Nginx/Apache are happy with the real cert.
Making this generic and automated is difficult (at best). It seems like 99.9% of all guides, blog posts, etc. about this problem are just like "start off with Nginx with a port 80 host... then after LE, switch it to 443" — or worse "let certbot create the 443 configs and just don't manage those in code/on your own" :O
Anyways, just wanted to put more notes here because this is the fifth time I've worked on automating everything end-to-end, so it's all in code, all reproducible (from nothing to server), etc.—and it's taken me hours each time. And I still can't get it done in a generic way I'd add to this role as a feature :(
Related upstream issue: https://github.com/certbot/certbot/issues/2933
Or just use dns validation through aws route 53 etc? :)
This is a hard problem because generating certs should be a seperate step and not part of the initial setup
Just make it a requirement that it has to be run multiple times to get the SSL cert fully installed. We don't have to solve all of the worlds problems in one Ansible run. One request thou, can we make the automated cert generation optional?
- stat:
path: /etc/letsencrypt/live/sitename.com/privkey.pem
register: prod_ssl
- name: prod_ssl exists
set_fact: prod_ssl_exists=true
when: prod_ssl.stat.exists
- name: adding sites-available
copy: src=apache2/sites-available/{{ item }} dest=/etc/apache2/sites-available/{{ item }} owner=root group=root mode=0644
with_items:
- site-name.conf
notify: restart apache2
when: prod_ssl_exists is defined
I implemented a standalone
setup for some CI servers I was building, and am working on getting that into this role first.
Then I'd like to make it so you can either use standalone (where Certbot runs it's own little service on ports 80/443 to respond during the cert generation process), or with a webserver/webroot (e.g. nginx or apache2, at least) like @oxyc showed earlier in this thread.
See: https://github.com/geerlingguy/ansible-role-certbot/pull/38 — I'm also going to add a playbook in the tests/
directory which would be a functional complete demonstration of how to use this role with Nginx (initially) and Apache to generate certs automatically.
Just about ready to wrap up the PR for this issue (https://github.com/geerlingguy/ansible-role-certbot/pull/38).
In the interest of moving issues in this queue forward, I'm going to close both this issue and https://github.com/geerlingguy/ansible-role-certbot/issues/6, and open a new issue for adding automatic certificate generation using the --webroot
option.
That will likely require a bit more testing and a more involved automated test playbook. Plus, I've run out of cert generation for my test domain (though I have about 80 other domains I can play with if I really need to...), so I'd like to tie this off at this point and refocus on making it so we have two cert generation methods:
I won't close this ticket until I open it's replacement.
Follow-up issue: https://github.com/geerlingguy/ansible-role-certbot/issues/39 (add a webroot
method).
Hey !
On first launch, I'd like to run the cerbot-auto command in order to download certificates and so. However, it's not possible to do it directly from ansible, I'm obliged to connect manually and launch the script.
Do you have any workaround for this ?
Thanks a lot !