Cannot Get Certificates

[ghost]

Recently trying to switch back to Caddy after my setup has altered enough to not need nginx anymore...anyway:

I cannot get any certificates at all:

{"level":"error","ts":1643740513.5424674,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"bib.actionsack.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for bib.actionsack.com (probably OK if presenting failed)"}
{"level":"error","ts":1643740513.5882883,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"bib.actionsack.com","issuer":"acme-staging-v02.api.letsencrypt.org-directory","error":"[bib.actionsack.com] solving challenges: presenting for challenge: adding temporary record for zone com.: expected 1 zone, got 0 for com. (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/42609938/1687218458) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}
{"level":"error","ts":1643740513.5883076,"logger":"tls.obtain","msg":"will retry","error":"[bib.actionsack.com] Obtain: [bib.actionsack.com] solving challenges: presenting for challenge: adding temporary record for zone com.: expected 1 zone, got 0 for com. (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/42609938/1687218458) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":2,"retrying_in":120,"elapsed":62.95889265,"max_duration":2592000}

This is an example for one subdomain. This happens for every single domain and subdomain (I have 2 domains and a ton of subdomains). In my global settings, I have:

{
  default_sni xnaas.info
  acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
  acme_dns cloudflare {$CLOUDFLARE_API_TOKEN}
  email {$ACMEEMAIL}
}

I was reading some other issues and tried setting Zone.Zone to Edit instead of Read, but that did not help.

I assume it's an API issue of some sort, since I don't see any TXT records being created...but unsure what the problem is. This same API key worked when I used Caddy many moons ago and has been working with nginx... 😅

Thoughts? Troubleshooting?

---

Edit: I've also tried setting a tls{} section specifying a resolver and such, but that caused more issues (400s and 403s), so that doesn't seem like the correct solution.

Edit 2: Actually, I switched to specifying this:

tls {$ACMEEMAIL} {
    ca https://acme-staging-v02.api.letsencrypt.org/directory
    dns cloudflare {$CLOUDFLARE_API_TOKEN}
    resolvers 1.0.0.1
  }

and now the logs are more like:

{"level":"error","ts":1643741780.2631567,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"dl.actionsack.com","issuer":"acme-staging-v02.api.letsencrypt.org-directory","error":"[dl.actionsack.com] solving challenges: presenting for challenge: adding temporary record for zone actionsack.com.: got error status: HTTP 403: [{Code:10000 Message:Authentication error}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/42609938/1687346048) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)"}
{"level":"error","ts":1643741780.2631755,"logger":"tls.obtain","msg":"will retry","error":"[dl.actionsack.com] Obtain: [dl.actionsack.com] solving challenges: presenting for challenge: adding temporary record for zone actionsack.com.: got error status: HTTP 403: [{Code:10000 Message:Authentication error}] (order=https://acme-staging-v02.api.letsencrypt.org/acme/order/42609938/1687346048) (ca=https://acme-staging-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":21.856070745,"max_duration":2592000}

So it's definitely an API issue, I guess...but I'm not sure why.

Edit 3: curl -X GET "https://api.cloudflare.com/client/v4/zones?name=actionsack.com" -H "Content-Type:application/json" -H "Authorization: Bearer <token>" works just fine, as well.

[rishubn]

I also encountered an issue today with cloudflare dns challenge. Are they related?:

WARN    tls.issuance.acme.acme_client   HTTP request failed; retrying   {"url": "https://acme-v02.api.letsencrypt.org/directory", "error": "performing request: Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org on 10.89.0.1:53: server misbehaving"}
2022/02/01 20:25:37.554 WARN    tls.issuance.acme.acme_client   HTTP request failed; retrying   {"url": "https://acme-v02.api.letsencrypt.org/directory", "error": "performing request: Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org on 10.89.0.1:53: server misbehaving"}
2022/02/01 20:25:37.806 WARN    tls.issuance.acme.acme_client   HTTP request failed; retrying   {"url": "https://acme-v02.api.letsencrypt.org/directory", "error": "performing request: Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org on 10.89.0.1:53: server misbehaving"}
2022/02/01 20:25:37.806 ERROR   tls.obtain      could not get certificate from issuer   {"identifier": "<wildcard domain>", "issuer": "acme-v02.api.letsencrypt.org-directory", "error": "registering account [] with server: provisioning client: performing request: Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org on 10.89.0.1:53: server misbehaving"}

[mholt]

@rishubn No, your DNS server at 10.89.0.1:53 is broken. ("server misbehaving")

[ghost]

Aha. It was a case of total user error. I had copied my fail2ban key and not my acme key (completely different perms). Sigh. I hate myself. :D