checkHTTP should sniff out CMS/app intercepting requests

alexzorin commented 6 years ago

It is common for people to send all requests to WordPress index.php or to proxy_pass every request to their Node app.

checkHTTP should return specific Problems when it thinks that is happening and provide concrete recommendations/links on how to configure your webserver or .htaccess to handle /.well-known/acme-challenge properly.

webprofusion-chrisc commented 6 years ago

I'm looking at the API just now and noticed the http-01 check queries ./well-known/acme-challenge/letsdebug-test - is there any expectation that the server might respond with an actual file result there or does a 404 count as success for this test?

We do a similar test in Certify (windows) which creates a configcheck file in the same folder, and an http 200 result counts as a pass. I'm thinking if we also created a letsdebug-test file that might be useful as well.

alexzorin commented 6 years ago

Hi Christopher,

The server response is largely disregarded except to detect a few things like problems with redirects (loops, bad locations).

It's possible that an enhancement to the API could serve your idea, for the client to nominate its own random response:

POST /
{"method":"http-01","domain":"example.org","expect-http-response": "<nonce>"}

which would then cause the test to require the correct answer to be present at each endpoint to get a pass.

If you think that would be valuable for your purposes I'd be happy to implement it.

webprofusion-chrisc commented 6 years ago

Hmm, yes that would be pretty useful! Perhaps the client could also specify the filename (so it can optionally be random as well, but definitely under .well-known/acme-challenge/ path)

I'm considering using the API as a fallback when our own configuration checks have failed and they want a second opinion :) - users will quite often we adamant that they're site is working fine (when it's only working for them).

Do you have a single server hosting these tests or is it something that scales? If so, I imagine that would start to cost a fair bit if lots of people use it?

alexzorin commented 6 years ago

Do you have a single server hosting these tests or is it something that scales? If so, I imagine that would start to cost a fair bit if lots of people use it?

Nah, costs are fixed and it's built to deal with pressure (realistically the first limit it's likely to hit are artificially imposed ones). I encourage you to send as many tests as you want. Just LMK if you submit tests from a single IP as there's an IP-based rate limit (HTTP 429).

Anyway, I have added the requested functionality. It's in the README.

curl --data '{"method":"http-01","domain":"whirlpool.net.au","options":{"http_request_path":"letsdebug-test-custom-path/haha","http_expect_response":"needs to match"}}' -H 'content-type: application/json' https://letsdebug.net

$ curl -sH 'Accept: application/json' https://letsdebug.net/whirlpool.net.au/2570 | jq .
{
  "id": 2570,
  "domain": "whirlpool.net.au",
  "method": "http-01",
  "status": "Complete",
  "created_at": "2018-07-16T10:06:17.575134Z",
  "started_at": "2018-07-16T10:06:17.587065Z",
  "completed_at": "2018-07-16T10:06:20.804195Z",
  "result": {
    "problems": [
      {
        "name": "ANotWorking",
        "explanation": "whirlpool.net.au has an A (IPv4) record (104.20.27.67) but a request to this address over port 80 did not succeed.",
        "detail": "This test expected the server to respond with \"needs to match\" but instead we got a response beginning with \"<html>\r\n<head><t\"\n\nTrace:\n@0ms: Making a request to http://whirlpool.net.au/.well-known/acme-challenge/letsdebug-test-custom-path/haha (using initial IP 104.20.27.67)\n@0ms: Dialing 104.20.27.67\n@553ms: Server response: HTTP 404 Not Found",
        "severity": "Error"
      },
      {
        "name": "ANotWorking",
        "explanation": "whirlpool.net.au has an A (IPv4) record (104.20.26.67) but a request to this address over port 80 did not succeed.",
        "detail": "This test expected the server to respond with \"needs to match\" but instead we got a response beginning with \"<html>\r\n<head><t\"\n\nTrace:\n@0ms: Making a request to http://whirlpool.net.au/.well-known/acme-challenge/letsdebug-test-custom-path/haha (using initial IP 104.20.26.67)\n@0ms: Dialing 104.20.26.67\n@513ms: Server response: HTTP 404 Not Found",
        "severity": "Error"
      },
      {
        "name": "CloudflareCDN",
        "explanation": "The domain whirlpool.net.au is being served through Cloudflare CDN. Any Let's Encrypt certificate installed on the origin server will only encrypt traffic between the server and Cloudflare. It is strongly recommended that the SSL option 'Full SSL (strict)' be enabled.",
        "detail": "https://support.cloudflare.com/hc/en-us/articles/200170416-What-do-the-SSL-options-mean-",
        "severity": "Warning"
      }
    ]
  }
}

I would really love to see an ACME client send traffic to this service, even as a fallback. Let me know how you go.

webprofusion-chrisc commented 6 years ago

Awesome! I'll check it out. I probably won't get it into the new 4.x release of Certify (this week) but it should be soon. We have a diagnostic testing UI built in and this would fit well with that as an additonal Full Diagnostic Test via LetsDebug (or something like that), as there is also some implied data sharing happening by calling the API so the user needs to know anyway.

letsdebug / letsdebug

checkHTTP should sniff out CMS/app intercepting requests #7