HTTP 403 from CloudFlare during DNS challenge

[shjryan]

Hello,

When I run: sudo caddy run -config TestCaddyfile -adapter caddyfile

I get this error: acme: error presenting token: got error status: HTTP 403: [{Code:0 Message:Actor 'com.cloudflare.api.token' requires permission 'com.cloudflare.api.account.zone.list' to list zones}]

My TestCaddyfile content is:

[domain] {
    tls {
         dns cloudflare [token]
    }
}

Token permission on CloudFlare:

When I run CURL using the token in bash, I can successfully get all the zone information.

Is there anything I can do to further debug where things are going wrong?

[mholt]

Hmm, it works for me (and many other people). I would just double-check that your token and permissions are indeed correct in your configs. (Feel free to continue discussion if this is in fact a bug, but first we have to be convinced it's a bug on our end.)

[shjryan]

@mholt Thanks for the response!

If I change the token even by one character, I get:

{"code":9109,"message":"Invalid access token"}

from CloudFlare, so I am assuming that the token in my caddyfile is correct.

Also I copied the token from the caddyfile into the CURL command, and it is able to retrieve zone list with this command:

curl -X GET "https://api.cloudflare.com/client/v4/zones?name=[mydomain.ca]" -H "Content-Type:application/json" -H "Authorization: Bearer [mytoken]"

Other than that, and along with the screenshot that I took from CloudFlare token page, where else could I look?

[mholt]

I believe your token is correct, but for some reason Cloudflare isn't convinced it has the right permissions, even though your screenshot clearly shows that it does.

Maybe there's a problem with the domain's configuration.

Feel free to have a look at our API request here: https://github.com/libdns/cloudflare/blob/16482ae4e8069b30e6ecef0b4b1fd69c28bd73a2/client.go#L90 -- if you see a problem with it, we can fix it, but like I said, it's working quite well, so 🤷

[shjryan]

I didn't see a problem with the libdns code either. That's why I am so confused why this is failing. Is there a more verbose mode?

There seems to some similar issues in CloudFlare community for people dealing with ACME challenge getting the same error like I am without any tangible solution. 🤦Gonna have to research more.

[mholt]

There's not currently, but if you have some time to print out the request that'd be super useful, since I can't reproduce the problem you're having!

And/or could accept a PR to enable a verbose mode, maybe a struct field or something. Edit: Actually, better yet, zap.Logger level of DEBUG.

[shjryan]

Ah... I was going to schedule some time this weekend to learn go and take a zap at this... but.... I solved the issue, and it was my fault.

I totally forgot that I had setup a dnsmasq rule to bind the domain to another NAT device on the network, and that was sucking up the requests, causing the request to fail... 🤦

[mholt]

It's always DNS 😄 Glad you figured it out!

[hmhackmaster]

@shjryan I stared at your description of what your issue was for several minutes before I understood how I could translate it for my scenario. Thank you for posting it, as it certainly led me down the right direction. I am commenting my findings here just in case someone else encounters the same issue and finds their way to this issue. And I'll be honest, I don't understand why it's occurring, as I don't think it should (and doesn't happen in other ACME clients). Me carving out time to wrap my head around it and figure that out will have to wait for another time.

My instance of Caddy (running v2.1.0) is running on a Debian VM inside a DMZ with it's DNS config pointed to an DNS forwarder running on my router. This router (a Mikrotik) is configured to forward DNS queries to my Windows Active Directory DNS servers (located in a different subnet). In this configuration, Caddy threw the following error when attempting to create a new DNS record for certificate creation: [test2.example.org] acme: error presenting token: got error status: HTTP 403: [{Code:0 Message:Actor 'com.cloudflare.api.token.REDACTED' requires permission 'com.cloudflare.api.account.zone.list' to list zones}]

Using curl, I was also successfully able to run the API call: curl -X GET "https://api.cloudflare.com/client/v4/zones?name=example.org" -H "Content-Type:application/json" -H "Authorization: Bearer CLOUDFLARE_API_TOKEN"

Re-configuring this VM (via /etc/resolv.conf) to talk directly to my AD DNS servers, or to any external public DNS resolvers, resolved the issue and resulted in the successful creation of records.

Based on all this, I presume that some DNS forwarders (like the one built into the Mikrotik RouterOS) don't support a particular record that Caddy/libdns use to get the record created in Cloudflare.

Why, I have no idea. But, as @mholt very accurately said, the issue is always DNS. 😄

[mholt]

@hmhackmaster Thank you for writing that up, I'm sure it will help other people who find this!