http-01 (rfc 8555 8.3) redirects to https fail validation as the certificate is invalid

[joejulian]

Subject of the issue

When the initial http-01 challenge is sent to port 80, it is valid to accept redirects. Often, ingress servers are configured to automatically redirect all connections to https on port 443. Since the certificate served by the server will not yet be valid until it is received via this process, the validation should set InsecureSkipVerify in http01Validate.

The RFC is somewhat conflicting in its wording:

Because many web servers allocate a default HTTPS virtual host to a particular low-privilege tenant user in a subtle and non-intuitive manner, the challenge must be completed over HTTP, not HTTPS.

yet

The server SHOULD follow redirects when dereferencing the URL. Clients might use redirects, for example, so that the response can be provided by a centralized certificate management server.

Let's Encrypt helped draft that RFC and their documentation states:

Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates (since this challenge is intended to bootstrap valid certificates, it may encounter self-signed or expired certificates along the way).

The HTTP-01 challenge can only be done on port 80. Allowing clients to specify arbitrary ports would make the challenge less secure, and so it is not allowed by the ACME standard. https://letsencrypt.org/docs/challenge-types/#http-01-challenge

So I believe that it is valid to follow their example.

Expected behaviour

A challenge should be successfully validated when the http-01 request, after connecting to port 80, is redirected to https on port 443.

Actual behaviour

The validation fails

[maraino]

Boulder also allows redirects to HTTPS (but only to port 443) using &tls.Config{InsecureSkipVerify: true}.

In boulder this creates the transport with the InsecureSkipVerify: https://github.com/letsencrypt/boulder/blob/5457680a9c8ce34d0456ccf289ed347a8529a31e/va/http.go#L489

This is the function that process the redirects: https://github.com/letsencrypt/boulder/blob/5457680a9c8ce34d0456ccf289ed347a8529a31e/va/http.go#L494-L565

And in `va.extractRequestTarget(req) they only allow port 80 or port 443: https://github.com/letsencrypt/boulder/blob/5457680a9c8ce34d0456ccf289ed347a8529a31e/va/http.go#L292-L297

Finally, the client with the transport and the check redirect method: https://github.com/letsencrypt/boulder/blob/5457680a9c8ce34d0456ccf289ed347a8529a31e/va/http.go#L567-L572

@dopey, for @joejulian PR we probably want to follow let's encrypt implementation of boulder and make sure that the only ports allowed are 80 or 443 in redirects.

[maraino]

The issues is partially fixed with #557 but we should only redirect to 80 and 443 ports.