Enable pluggable certificate generation

[mgilbir]

In the same way that Caddy provides support for Let's encrypt! and self-signed certificates as discussed on #327, it would be great if those two methods were abstracted behind an interface that would enable the addition of other certificate providers that expose an API.

A concrete use case I have is getting certificates generated from a vault server. Right now I have a script that gets the certificates, puts them in the correct place and restarts caddy, but all cert renewals have to be handled outside of caddy.

[mholt]

Hey @mgilbir, thanks for the feature request!

With Caddy 0.9 it will be possible to add plugins that do any variety of things, such as add hooks to the GetCertificate callback, which is the function Go executes during the TLS handshake to get a certificate. I'm not sure how this would work, though. I guess we call all the registered hooks/plugins and if they don't return a certificate, we just call our own?

(See #188)

[groob]

I realized that this issue is about server certificates, but...

I'm working on a SCEP server for Go and hoping to add a caddy middleware package for it.

At the moment, the SCEP server signs and manages certificates itself, but in the future it will support using the Vault or cfssl backends for signing certs and just provide them to devices through the SCEP protocol. My use case is iOS and OS X devices, which have a supported method of requesting certificates throgh SCEP and storing them in the keychain.

https://github.com/micromdm/scep

[mholt]

So, @groob, would the ability to hook into the GetCertificate callback be helpful to you for that use case?

[mholt]

Looking at this again... @mgilbir Which one would you prefer -- plugging into the startup phase where certificates are generated, or plugging into the actual handshake-time function that loads a certificate to give to a client?

[mgilbir]

I'm not sure I'm understanding the difference between the two options you are proposing. Sorry!

As a user, this is what I had in mind:

1. Server starts
2. If it doesn't have a valid certificate, requests one with the configured provider (eg. vault, SCEP server, let's encrypt, ...) if exists or a suitable default (eg. let's encrypt for public sites, self-signed for non-public sites) if it doesn't.
3. Serves happily requests with that certificate until a renewal is needed. Renewal is defined as some absolute time duration before expiration, or percentage of total validity, ...
4. When renewal of the certificate is needed, it repeats from step 2

Does it make things clearer?

[mholt]

@mgilbir Yes, let me rephase my question now. :smile:

The question is whether step 2 happens when you first run the server (before the socket is opened) or during the TLS handshake (when a client is actually connecting). Caddy can do both/either. Which one did you want to plug into?

[mgilbir]

I may be biased by the type of systems I work with but, personally, I'd rather do it before the socket is opened. My rationale is that, once I open the socket to start accepting connections the clients can be serviced immediately. If you wait until the first client comes in, that first request and any other that arrives before the certificate is available will incur in an extra latency. For a high traffic server that may affect quite a few requests.

What's the use case for waiting until the first client comes in?

[mholt]

Startup-time it is then!

The handshake-time cert loading is a feature of Caddy called on-demand TLS. It makes it easy to scale your service and update your TLS configuration on-the-fly with no downtime.

[jwkohnen]

Offtopic, but maybe of interest: After 0.9 has landed, I'd like to implement some ideas that I have regarding HPKP. For that I want to plug in to the way Caddy generates keys (not certs).

[mholt]

@wjkohnen Good to know. I'll consider that... although you will probably want to see how xenolf/lego does it, since that library's acme package is what we use to generate keys for certificates.

Note that neither Caddy nor the acme package generate a new key when renewing a certificate; it reuses the same key until it is deleted or revoked.

[jwkohnen]

TBH, I haven't looked into your changes of 0.9 and I expect that in the worst case I may have to do a lot of plumbing with lego. HPKP is technically trivial, but it's modeling all the "action paths" in respect of desaster recovery et cetera that is challenging. I don't yet completely know what all that means on the technical level.

Edit: What I do know is, it needs off-site tooling.

[mholt]

@wjkohnen If this doesn't make it into Caddy 0.9, bug me about it for Caddy 0.9.1 or something.

You can see the 0.9 stuff in a real working branch here, it will soon be merged into master.

[mholt]

@mgilbir Would you mind proposing a specific implementation that you would find helpful? You may find this wiki page about extending Caddy helpful so you can get an idea of how plugins are registered.

[mgilbir]

Sure. I'll take a look over the weekend On 20 Jun 2016 10:49 p.m., "Matt Holt" notifications@github.com wrote:

@mgilbir https://github.com/mgilbir Would you mind proposing a specific implementation that you would find helpful? You may find this wiki page about extending Caddy https://github.com/mholt/caddy/wiki/Extending-Caddy helpful so you can get an idea of how plugins are registered.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mholt/caddy/issues/860#issuecomment-227249281, or mute the thread https://github.com/notifications/unsubscribe/AAKOpERlj-0JAQN0pAO0TACAJZiC-Fk9ks5qNu68gaJpZM4IsZvp .

[mholt]

Did you have a chance to look at that @mgilbir? I want to get Caddy 0.9 out of beta soon. :)

[mholt]

So we're actually implemented a certificate storage plugin system, I think that will solve this problem, since Caddy will ask the storage provider for a certificate -- and if it returns one, it will just use that. Note that it will also be used to store certificates when it obtains new ones from ACME. This storage provider interface also supports locking for sharing across multiple instances of Caddy. Will that be sufficient for what you want?

[mholt]

Okay, so re-reading a bit more... I see that this may not suit your needs. What we're working on is storage only for managed TLS, so if your certificates can't be renewed with ACME then it won't be a good fit.

We could look into a way to make the "Obtain" and "Renew" methods part of an interface. Would be happy to have your suggestions @mgilbir !

[mgilbir]

I'm finally back after some time busy with work, family and holidays. Sorry it took so long!

I've been taking a look at the storage plugin system and the caddytls package in general. Still trying to figure out how everything comes together in the configuration and what gets called when.

My initial thoughts for the particular case of the vault integration I would like to have is that it may be possible to make it behave as a storage plugin. Nevertheless, after a quick look it is not clear to me how I can pass configuration parameters from the config file into the vault storage plugin without changing things too much.

That led me to take a look around the caddytls package and caddytls.Config in particular to see if there was another way to hook this up. The caddytls.Config struct feels very ACME-centric. I understand this may be by design but reading the code I had the feeling that it was missing an abstraction that could make it really easy to add other providers.

For example, while I understand that DNSProvider and StorageProvider serve different purposes, aren't they in they end responsible for providing a certificate to the caddy server? If so, there may be an interface that those providers could satisfy (maybe the Obtain and Renew methods that you mentioned). The current behavior could be made to implement that interface and this would allow composing them in whatever crazy ways the user may want (eg. if cert does not exist in etcd storage, use ACME with DNS and put retrieved cert in the etcd storage).

If I should take a look at other parts of the projects to see where things could get hooked or if I missed something obvious in caddytls, please let me know :)

[mholt]

Hey @mgilbir - thanks for taking a look! Abstracting ACME away is an interesting idea, and I think we can do that if we move the ACME logic into a type that satisfies an interface that can Obtain and Renew. Maybe. I'm not sure what the signatures would look like yet, but that's something we can work on.

DNSProvider and StorageProvider are different than this new interface, which I might call something like CertificateClient. I think we will still need to separate the notions of storage, DNS challenge solvers, and cert clients.

So some refactoring may be in order. I'll look into it when I get a chance!

[mholt]

@mgilbir You know, for now, you could just swap out calls to newACMEClient (<-- like that one) with something that has the same signature, and voila, you can do what you need, I think.

When you do that, let me know how it goes, and we can look at generalizing it if it's useful.

[Nitecon]

Does caddy make use of dns challenge hooks for TLC cert requests against lets encrypt, it's a new feature but something that will make many things much easier? Biggest blocker for us is our QA / UAT environments are firewalled off so we either have to use no https / caddy or use native go with standard csr generated certs. Since all of our environments match 100% we put ssl in lower envs too. What DNS hooks will allow is to have letsencrypt do validation against your dns provider to validate instead of against the server itself, so you could have certs in lower environments.

To also add to time of when regeneration happens should be before socket open, on super heavy clients where a new server gets introduced into an auto scaling group that can have a lot of negative impact to first have to do a lookup after the server has gone live. I assume that expiration tracking is in place and you don't have to restart caddy to get a new cert?

[mholt]

@Nitecon

Does caddy make use of dns challenge hooks for TLC cert requests against lets encrypt, it's a new feature but something that will make many things much easier?

I don't know what a "TLC cert request" is -- sorry. :( Can you link me to something that describes that?

Biggest blocker for us is our QA / UAT environments are firewalled off so we either have to use no https / caddy or use native go with standard csr generated certs.

This sounds like you want the DNS challenge.

I assume that expiration tracking is in place and you don't have to restart caddy to get a new cert?

Correct.

I think we're getting off-topic here, though, if you have further questions about Caddy's TLS features feel free to post in our forum!

[Nitecon]

@mholt First paragraph I put in was in regards to dns challenge hooks, is there support for it in the current caddy implementation or is it just the standard call for validation against the public server? Sorry had typos in there. I don't see anything in docs regarding this although I may have overlooked it.

Thanks!

[mholt]

Did you read the rest of my post? ;)

[Nitecon]

omg i'm so sorry I thought you linked to the LetsEncrypt spec for dns challenge doh. Thanks for the polite answers though 👍

[mholt]

@mgilbir It was fun to meet you at dotGo! Here's some more help to get you started. If this succeeds, we can turn it into a new plugin type.

Replace the call to newACMEClient() on these lines:

• https://github.com/mholt/caddy/blob/bedad34b25cfcc67620fa5b854fed020547efbdf/caddytls/config.go#L149
• https://github.com/mholt/caddy/blob/bedad34b25cfcc67620fa5b854fed020547efbdf/caddytls/config.go#L159
• https://github.com/mholt/caddy/blob/bedad34b25cfcc67620fa5b854fed020547efbdf/caddytls/tls.go#L70

with a function that you write, say, newVaultClient() or something, that returns a value which implements this interface:

type CertificateManager interface {
    Obtain(name string) error
    Renew(name string) error
    Revoke(name string) error
}

The one thing that I'm not quite sure about from this vantage point is you'll have to integrate with the storage for where the certs are located so that you can place them there once you've obtained them. Let me know how it goes!

[mholt]

This has been implemented in Caddy 2 (and is already available in CertMagic today).