Disable rotation of the private key

[Vcele]

I am using caddy as a reverse proxy for mailcow, and I am searching for a way to disable private key rotation on renewal of the ssl certificate so that I don't have to update my TLSA records.

[mholt]

This is configurable in CertMagic, I just haven't exposed an option to do it in Caddy yet. Should be easy enough, though I generally advise against pinning keys.

[Lan-Hekary]

Hello, @mholt I am also looking for a way to pin the private key, and/or supply the CSR to caddy. Also I need a way to get a single certificate for multiple sub domains, not a wild card. I guess let's encrypt support that, but i's not possible inside caddy.

Key Pinning is Bad, but is some cases it's make sense to use it as long as you keep your private key secure. Also the default setting would be to rotate it, so most of the users who does not care would not change it.

[mholt]

There is no good reason to use a multi-SAN certificate in automation other than extremely limited storage space (never heard of that), or paying per certificate. When you're using Caddy multi SAN certs are only harmful. (But that is off topic for this issue.)

Anyway, as for private key rotation, I will think about making it configurable in Caddy, but it goes against my better judgment as, like you said, key pinning is not really a best practice.

[Lan-Hekary]

Well you are right, it's a storage space issue, for me its embedded systems like IoT devices. The resources are limited, Storage, RAM, CPU and Battery power are all limited. Certificate validations takes up a good chunk of connection time. Also connections to multiples sub-domains require either one common root CA, which increase validation time for every connection. Or multiple Certificates, more storage and the hassle of changing the certificates every renewal. Anyway, it's off topic like you said, but I just wanted to give you an example for a use case.

There is an option to use public key, which bypass the signature validation, and it's still secure as long as the private key is secure. And Multiple certificates can be issued with the same private key. So that will be a good solution for the Low Power Version.

[mholt]

So just to make sure I understand, you have embedded IoT devices serving thousands of domain names each? 🤔

Certificate validations takes up a good chunk of connection time.

What do you mean by this exactly? Do you mean the low powered devices are slow at completing the TLS handshakes?

Also connections to multiples sub-domains require either one common root CA, which increase validation time for every connection. Or multiple Certificates, more storage and the hassle of changing the certificates every renewal.

I'm not sure I follow this. Subdomains are orthogonal to root CAs used.

[Lan-Hekary]

1. Not serving, making HTTPS calls to distributed servers, not thouthands 😂 but a few, I use caddy as a reverse proxy. Caddy server is NOT on the IoT Devices, but they make calles to it.

2. Yes, they are very slow, a TLS handshake can take between 200ms and 6s depending on the cipher suite used, clock speed also matters but higher clock speeds means more power draw. The problem here is that The MCUs does not have hardware accelerations for the Crypto operations. Some Operations are Faster, like RSA is faster than ECDSA.

3. Yes, the Perfect ways is to store a the Root CA on the devices and validate the entire Chain of trust every connection, but that takes time. Because the device has to download the CA Chain, validate the intermediate, then validate the Leaf, the move on with key exchange and so on. Every signature validation takes time.

I can compare again the leaf certificate, or the the hash of the certificate but if I am using let's encrypt it will expire and I have to update it every 3 month, it's not a realistic solution anyway. Also I have to store a certificate for each subdomian, lots of storage required.

If I pin the public key in the certificate, I can skip the certificate validation all together without affecting security, because only the server that have the private key can complete the handshake.

I can have multiple domains withe multiple certificates using the same private key, one certificate with multiple SANs using a single private key, or even a wildcard. it's the same for me. Because I can add the public to the IoT devices and make Connections faster without the need for expensive validations for multiple domains.

On that note, there one more thing I want to ask you. Are there anyway to support session resumption through Session ID in caddy? I am using TLS 1.2 stack you can check it here https://bearssl.org And it support session ID resumption, but not Tickets.

[mholt]

@Lan-Hekary

Session resumption is risky, so it is obsolete with TLS 1.3. I am not sure there is any interest from the Go team implementing support for high-risk features in an old version of the protocol: https://github.com/golang/go/issues/25228

Let's get your company set up on a sponsorship and we can work on a solution for you. Feel free to book a time at https://matt.chat to discuss!

[rithvikvibhu]

Hey, I'm in need of the same config (forcing key reuse for TLSA). Is there anything I can do to help add this? @mholt

[mholt]

We could review a PR to implement it. I don't have time right now (without a sponsorship for it) since we just had a baby, but I can probably review a simple PR.

[mholt]

Thinking on this more, a better solution that I'd be more likely to accept is to have a DNS API update the TLSA records when the private key changes. That's something I'd willingly add support for (with time and/or a sponsorship).

[Lan-Hekary]

Hello @mholt,

Can you expose the API to disable private key rotation to us in Caddy, and maybe add a warning in the logs? You said it should be easy.

This is configurable in CertMagic, I just haven't exposed an option to do it in Caddy yet. Should be easy enough, though I generally advise against pinning keys.

[mholt]

It is, but I don't believe it's the Right Thing to Do :tm:. (See my last comment.)

[rithvikvibhu]

a better solution that I'd be more likely to accept is to have a DNS API update the TLSA records when the private key changes

That would be ideal! The same libdns set of plugins can be used here, but iirc many of them are only tested for TXT records, not other types. But not a big deal, they might just need some small changes.

If caddy is going to handle updating TLSA, that brings up a few questions regarding how rollovers are handled. With DNS the way it is, TLSA records must be added, then after a while, the new cert starts being served (and then the old TLSA record is removed).

And just to clarify, while I'd love to see TLSA auto updates, does this mean that the private key reuse option won't be added / PR accepted? It's a much simpler change and with defaulting to off, imo it would still be nice to have.

[y8]

Sadly, updating the TLSA record is a risky and tricky business because of, uh, DNS.

For example, the recommendations for DANE suggest publishing both the current and future records for at least 2 × TTL durations before changing the actual certificate presented by the server.

This is crucial because if the certificate changes before the client can resolve the updated TLSA record, the client will face all the TLSA mismatch consequences, which is bad, like in "paging all responders for critical level incident" bad.

With DNS, we basically have zero control over record expiration in non-authoritative DNS server caches. Nowadays, this chain of caches is quite long: users' local DNS cache, which uses WiFi routers DNS servers, which in turn use ISP's DNS servers, which might or might not use some upstream DNS server (like 1.1.1.1 or 8.8.8.8). Any of these DNS caches might not respect TTL to save bandwidth, reduce latency, or for some other weird reasons.

So, if the TLSA doesn't match the current certificate, there is no "rolling change back", so you can't do anything about it, other than watching how your service is unaccessible to an unpredictable number of clients for an unpredictable period of time.

As per NIST SP 800-57, part 1 "Recommendation for Key Management: Part 1 – General" (Table 1, page 45), the recommended key material lifetime (cryptoperiod) is measured in years. A bit earlier, in section "5.3.3.2 Cost of Key Revocation and Replacement," they discuss that the complexity of key replacement is a valid reason for lifetime extension. I would say that cost of this DNS stuff is pretty high.

In my opinion, having an option to just keep the private key is a more feasible option than implementing proper TLSA rotation, at least at the short term.

[mholt]

If caddy is going to handle updating TLSA, that brings up a few questions regarding how rollovers are handled. With DNS the way it is, TLSA records must be added, then after a while, the new cert starts being served (and then the old TLSA record is removed).

Multiple records? Append the new one before rotating, allow some time for it to propagate, then rotate the key, then delete the old record. This is not trivial but is probably best.

@y8 Noted; and I'm not ruling it out completely, but we have strong aversions to key pinning after prior experience.