HTTP-01 challenge behind cloudflare improvement

[BioEvo]

1. What would you like to have changed?

Add a new method for HTTP-01 challenge(like tls HTTP-01_over_self_signed), it takes four steps:

1. set SSL/TLS encryption from "strict" to "full" in the cloudflare panel.
2. start the new site (hostname.domain) with a self_signed certificate.
3. just use HTTP-01 challenge to obtain unauthorized from Let's encrypt.
4. set SSL/TLS encryption back to "strict", as caddy always renew certificate in last 30 days, HTTP-01 challenge will always be working.
   • Automatic HTTPS Rewrites is required in the cloudflare panel.

2. Why is this feature a useful, necessary, and/or important addition to this project?

tls-alpn-01 challenge requires to expose origin server's ip to the internet, which might cause real ip leaking. dns challenge requires API that saved in environment, which meas it would be impossible to use different API KEY.

3. What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

No, but in current situation, HTTP-01 will be more tricky to work with clouldflare:

1. open port 80 for caddy's HTTP-01 challenge.
2. set SSL/TLS encryption to "Flexible" in the cloudflare panel.
3. start the new site (hostname.domain) with "tls yourname@domain".
4. waiting for tls-alpn-01 failure and fall back to HTTP-01, then get certificated.
5. close port 80 and disable http in caddy.
6. set SSL/TLS encryption to "strict".

4. Please link to any relevant issues, pull requests, or other discussions.

[BioEvo]

Although HTTP-01 challenge can only be done via http, Let's encrypt follows redirects, which means Let's encrypt supports HTTP-01 challenge under cloudflare's "strict" mode and "flexible" mode. Even without this feature, caddy has no difficulty to renew existing certificate once a https site has been certificated.

The only problem is HTTP-01 challenge under cloudflare's "flexible" mode might cause "too many redirects" for other https sites of the same host, that's why HTTP-01_over_self_signed is batter.

The HTTP-01 challenge can only be done on port 80. Allowing clients to specify arbitrary ports would make the challenge less secure, and so it is not allowed by the ACME standard.

Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates (since this challenge is intended to bootstrap valid certificates, it may encounter self-signed or expired certificates along the way).

[whitestrake]

What I read from this request is that you would like Caddy to be able to solve HTTP-01 challenges if they come in over HTTPS, by negotiating that HTTPS connection with a self-signed certificate. This would allow Caddy to handle first-time HTTP-01 validation through reverse proxies that connect over HTTPS regardless of the client's scheme (i.e. Cloudflare).

I'm sure it's doable, but I don't think in the long run it makes your use case much less trickier at all. Much of that work still needs to be done manually.

Honestly, I'd propose that the simplest solution is simply to acquire a long life origin certificate from Cloudflare. Set up mutual auth on your sites as well if you like, Cloudflare publish their CA for authenticated origin pulls. Doesn't get much more secure, or much less effort in the long run, than that (outside of ditching Cloudflare's proxy).

[BioEvo]

What I read from this request is that you would like Caddy to be able to solve HTTP-01 challenges if they come in over HTTPS, by negotiating that HTTPS connection with a self-signed certificate. This would allow Caddy to handle first-time HTTP-01 validation through reverse proxies that connect over HTTPS regardless of the client's scheme (i.e. Cloudflare).

I'm sure it's doable, but I don't think in the long run it makes your use case much less trickier at all. Much of that work still needs to be done manually.

Honestly, I'd propose that the simplest solution is simply to acquire a long life origin certificate from Cloudflare. Set up mutual auth on your sites as well if you like, Cloudflare publish their CA for authenticated origin pulls. Doesn't get much more secure, or much less effort in the long run, than that (outside of ditching Cloudflare's proxy).

Yeah, the simplest and stable solution is simply to acquire a long life origin certificate from Cloudflare. Meanwhile HTTP-01_over_self_signed(or HTTPS-01 for short) works fine automatically behind cloudflare's "full" mode without other manually operation(unless they want to do it under "strict" mode), this feature might be a great help for the beginner.

[mholt]

We could look into accommodating this use case more in Caddy 2. It's not a hugely high priority right now to be honest, but I'd welcome design docs/discussion!

[BioEvo]

Thanks for your reply! I'am sure this feature can't be a high priority since there are too many works to do with Caddy2.

[harningt]

From my understanding of your use case, you want all traffic to your caddy server to be through Cloudflare.

I this case, you want to use Cloudflare's origin ca setup. They will issue a certificate up to 15yrs chaining to their special CA that they recognize. This gets your traffic encrypted to them... but doesn't block any browsers accessing the original site - they would just get trust warnings.

You would also want to update firewall rules to only allow traffic from Cloudflare's IP list (more details at https://support.cloudflare.com/hc/en-us/articles/201897700-Whitelisting-Cloudflare-IP-addresses )

[ryanjaeb]

You can use http-01 through Cloudflare if you're able to open port 80 to your origin server. I'm doing something similar. I use Traefik and haven't tested it with Caddy, but I think it would work the same since it's pretty generic.

Disable Cloudflare's Always Use HTTPS option and configure your origin server do the http --> https redirects. Alternatively (or additionally) you can add 2 Page Rules to enforce the redirect on Cloudflare.

http://*example.com/.well-known/acme-challenge/*
SSL: Off, Cache Level: Bypass

http://*example.com/*
Always Use HTTPS

Those rules need to be in the order listed. The first one allows http-01 challenges to be proxied to your origin server via http. The second redirects all other requests to https. The wildcards match the bare domain and all subdomains.

You can leave Cloudflare in Full (strict) mode right from the start.

Based on my experience, you can use the same strategy to get Cloudflare in front of almost any service where you want to obscure the origin as long as the service supports http-01 challenges. I pointed the domain I was testing with tonight at GitLab Pages to demonstrate.

echo -n \
    | openssl s_client -connect nullpipe.one:443

Certificate chain
 0 s:C = US, ST = CA, L = San Francisco, O = "Cloudflare, Inc.", CN = sni.cloudflaressl.com
   i:C = US, ST = CA, L = San Francisco, O = "CloudFlare, Inc.", CN = CloudFlare Inc ECC CA-2
 1 s:C = US, ST = CA, L = San Francisco, O = "CloudFlare, Inc.", CN = CloudFlare Inc ECC CA-2
   i:C = IE, O = Baltimore, OU = CyberTrust, CN = Baltimore CyberTrust Root

GitLab is configured to automatically obtain certificates via Let's Encrypt.

echo -n \
    | openssl s_client -connect nullpipe.gitlab.io:443 -servername nullpipe.one

Certificate chain
 0 s:CN = nullpipe.one
   i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
 1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
   i:O = Digital Signature Trust Co., CN = DST Root CA X3

[mholt]

Thanks for the discussion -- feel free to continue it; but as I don't think the current proposal is specific enough or a good fit for Caddy at this point (maybe it is a better fit for an ACME-related dependency?), I will close the issue unless something more actionable arises from it.