caddyserver / caddy

Fast and extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS
https://caddyserver.com
Apache License 2.0
55.59k stars 3.91k forks source link

Embedded ACME proxy server? #3539

Open BioEvo opened 4 years ago

BioEvo commented 4 years ago

This allows other ACME clients (like Caddy under privite networks) to use another Caddy as an ACME endpoint, and proxy/convert ACME challenge to upstream like Let's Encrypt.

pinkeen commented 3 years ago

I love the idea as it would allow to avoid having to provision client nodes' trust store with the private CA certificates. This is very convenient as this setup (and later maintenance) can become really complex, especially in big heterogenous environments, embedded devices, etc.

I'd like to add some random input (rambling) to this request too.

Certificate cache

Cache all certificates returned on behalf of the ACME upstream to prevent failures that would be caused by rate limits of the upstream ACME authority.

Possible use-case scenario from real-life experience: Stateless and/or ephemeral cloud nodes which need to request a new certificate on each start, often for the same domains.

Custom authentication challenge (zero-configuration?)

Authenticate the requester by verifying the trust relationship inferred from existing configuration - e.g. requester is an upstream with known client certificate.

I don't have any good idea for verifying the authorization to receive specific certificate though. I wonder if step-ca's provisioning providers could be used for this... Possibly, if the requester is already an upstream and the requested domain matches any of the the ones that it already handles traffic from then there is no challenge needed at all.


Setup I have that would benefit from this feature:

¹ The ingress is connected to the upstreams via Nebula Overlay Mesh Network. ² There might be multiple private instances of course, but in my case at most two or three (per virtualization host I guess) as it's usually much less work to set up and manage (monitor) a single caddy instance that to configure each service separately to use acme for its certs. Especially that these services are usually running on very different hardware/virt and software platforms.

PS On an unrelated topic - I've just come up with a crazy but vague idea. Nebula is golang too, also uses PKI authentication and has a built-in DNS server (which I actually use as a sub-zone of my primary domain's zone). It would be super-cool to marry the two together. Not sure how though - maybe one could partially automate or provide services for the other so common-usage scenarios could be simplified using one piece of software. Huh, this would be kind of an HTTP routing mesh with out-of-the-box private overlay network.


👆 I've just realized how utterly pointless my setup is. 😞 I can just use nginx on the public node as it can pass-through encrypted traffic while still retaining the ability to route it based on SNI Host (ngx_stream_ssl_preread_module. I've even done this before... Though it's not possible to do any custom handling of the encrypted traffic on the ingress server though so I cannot use a singe authorization portal for all of the upstreams (planned to use caddy for this).

mholt commented 3 years ago

@pinkeen Very interesting, thanks for writing that up.

PS On an unrelated topic - I've just come up with a crazy but vague idea. Nebula is golang too, also uses PKI authentication and has a built-in DNS server (which I actually use as a sub-zone of my primary domain's zone). It would be super-cool to marry the two together.

If it can be used as a library, it can probably become a Caddy module without too much trouble! https://caddyserver.com/docs/extending-caddy

I can just use nginx on the public node as it can pass-through encrypted traffic while still retaining the ability to route it based on SNI Host (ngx_stream_ssl_preread_module

Caddy can do this too with Project Conncept that I'm working on (it already works, actually), a layer 4 proxy: https://github.com/mholt/conncept

francislavoie commented 3 years ago

Caddy can do this too with Project Conncept that I'm working on (it already works, actually), a layer 4 proxy: mholt/conncept

For whoever reads this, this is now publicly available: https://github.com/mholt/caddy-l4

m-martin-78 commented 10 months ago

Hi, I love the idea of an embedded ACME proxy server.

For now, I'm trying to use this project: https://github.com/noahkw/acmetk

However, I ran into an error with Caddy as a client of this ACME proxy:

2023-08-18 16:42:47,954 - aiohttp.access - INFO - 10.0.2.100 [18/Aug/2023:14:42:47 +0000] "POST /new-order HTTP/1.1" 400 389 "-" "Caddy/ CertMagic acmez (linux; amd64)"
[... stacktrace snipped ...]
josepy.errors.DeserializationError: Deserialization error: Could not decode 'status' (''): Deserialization error: Status not recognized

It looks like the Caddy ACME client, when ordering a new certificate, does not set the "status" property to a valid value (see https://datatracker.ietf.org/doc/html/rfc8555#section-7.1.3)

What do you think?

(related ticket on the original project: https://github.com/noahkw/acmetk/issues/64)

mholt commented 10 months ago

@m-martin-78 I would be surprised if it's a bug in the ACME client (ACMEz) due to it being used successfully without bugs in production for years now. But maybe? :man_shrugging: It doesn't really make sense though, as the client doesn't set the status when requesting a certificate. The server's job is to set the status, and the client uses that to control its behavior.

Unfortunately the error message doesn't really give us much to go on. If you can make it minimally reproducible (without an ACME proxy) then please file an issue at https://github.com/mholt/acmez with instructions to minimally reproduce the issue.

(And let's keep this issue on-topic.) Thanks!