Creating multiple _acme-challenge entries in cloudflare

[TheLinuxGuy]

Hi, thank you for your script. I am trying to use Cloudflare with DNS API key and it is failing at "DNS record propagation" timeout.

Checking my cloudflare control panel shows two (2) TXT _acme-challenge entries with different values. This seems like a bug.

root@sin:~# /usr/local/bin/synology-letsencrypt.sh
2024/02/28 00:22:00 [INFO] [mydomain.com, *.mydomain.com] acme: Obtaining bundled SAN certificate
2024/02/28 00:22:01 [INFO] [*.mydomain.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/320337866217
2024/02/28 00:22:01 [INFO] [mydomain.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/320338810707
2024/02/28 00:22:01 [INFO] [mydomain.com] acme: Could not find solver for: tls-alpn-01
2024/02/28 00:22:01 [INFO] [mydomain.com] acme: Could not find solver for: http-01
2024/02/28 00:22:01 [INFO] [mydomain.com] acme: use dns-01 solver
2024/02/28 00:22:01 [INFO] [*.mydomain.com] acme: authorization already valid; skipping challenge
2024/02/28 00:22:01 [INFO] [mydomain.com] acme: Preparing to solve DNS-01
2024/02/28 00:22:02 [INFO] cloudflare: new record for mydomain.com, ID 60b1f5eee519cdf4d3523eddaee709d6
2024/02/28 00:22:02 [INFO] [mydomain.com] acme: Trying to solve DNS-01
2024/02/28 00:22:02 [INFO] [mydomain.com] acme: Checking DNS record propagation using [192.168.20.1:53 192.168.20.1:53]
2024/02/28 00:22:04 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s]
2024/02/28 00:22:04 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:06 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:08 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:10 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:12 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:14 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:16 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:28 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:30 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:33 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:35 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:37 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:39 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:41 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:43 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:45 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:47 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:49 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:51 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:53 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:55 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:57 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:22:59 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:21 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:23 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:25 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:27 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:29 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:31 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:34 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:36 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:38 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:40 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:42 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:44 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:46 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:48 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:50 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:52 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:54 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:56 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:23:58 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:24:00 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:24:02 [INFO] [mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:24:04 [INFO] [mydomain.com] acme: Cleaning DNS-01 challenge
2024/02/28 00:24:05 [INFO] Skipping deactivating of valid auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/320337866217
2024/02/28 00:24:05 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/320338810707
2024/02/28 00:24:05 Could not obtain certificates:
    error: one or more domains had a problem:
[mydomain.com] propagation: time limit exceeded: last error: NS eric.ns.cloudflare.com. did not return the expected TXT record [fqdn: _acme-challenge.mydomain.com., value: RFvE0jCQHfFDEuPOxkSQ_1L-L64CfQm0Wrm2UoVrLqk]: L6ai0iWxb7i3P0nU8YqBhbWxrkv9nDJ8SDp-LVOr9xA
root@sin:~# nano /usr/local/etc/synology-letsencrypt/env
root@sin:~# /usr/local/bin/synology-letsencrypt.sh
2024/02/28 00:25:11 [INFO] [sin.mydomain.com, *.sin.mydomain.com] acme: Obtaining bundled SAN certificate
2024/02/28 00:25:12 [INFO] [*.sin.mydomain.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/320339644127
2024/02/28 00:25:12 [INFO] [sin.mydomain.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/320339644137
2024/02/28 00:25:12 [INFO] [*.sin.mydomain.com] acme: use dns-01 solver
2024/02/28 00:25:12 [INFO] [sin.mydomain.com] acme: Could not find solver for: tls-alpn-01
2024/02/28 00:25:12 [INFO] [sin.mydomain.com] acme: Could not find solver for: http-01
2024/02/28 00:25:12 [INFO] [sin.mydomain.com] acme: use dns-01 solver
2024/02/28 00:25:12 [INFO] [*.sin.mydomain.com] acme: Preparing to solve DNS-01
2024/02/28 00:25:12 [INFO] cloudflare: new record for sin.mydomain.com, ID b2f927b228486f8d29365bb9f9866830
2024/02/28 00:25:12 [INFO] [sin.mydomain.com] acme: Preparing to solve DNS-01
2024/02/28 00:25:12 [INFO] cloudflare: new record for sin.mydomain.com, ID a1c92b8cded7d97496e01563d47fbcb9
2024/02/28 00:25:12 [INFO] [*.sin.mydomain.com] acme: Trying to solve DNS-01
2024/02/28 00:25:13 [INFO] [*.sin.mydomain.com] acme: Checking DNS record propagation using [192.168.20.1:53 192.168.20.1:53]
2024/02/28 00:25:15 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s]
2024/02/28 00:25:21 [INFO] [*.sin.mydomain.com] The server validated our request
2024/02/28 00:25:21 [INFO] [sin.mydomain.com] acme: Trying to solve DNS-01
2024/02/28 00:25:21 [INFO] [sin.mydomain.com] acme: Checking DNS record propagation using [192.168.20.1:53 192.168.20.1:53]
2024/02/28 00:25:23 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s]
2024/02/28 00:25:29 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:31 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:33 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:35 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:42 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:44 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:46 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:48 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:50 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:52 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:54 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:56 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:25:58 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:00 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:02 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:04 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:07 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:09 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:11 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:13 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:15 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:17 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:29 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:31 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:33 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:35 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:37 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:39 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:41 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:43 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:45 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:48 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:50 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:52 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:54 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:56 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:26:58 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:27:00 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:27:02 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:27:14 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:27:16 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:27:18 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:27:20 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:27:22 [INFO] [sin.mydomain.com] acme: Waiting for DNS record propagation.
2024/02/28 00:27:24 [INFO] [*.sin.mydomain.com] acme: Cleaning DNS-01 challenge
2024/02/28 00:27:25 [INFO] [sin.mydomain.com] acme: Cleaning DNS-01 challenge
2024/02/28 00:27:25 [INFO] Skipping deactivating of valid auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/320339644127
2024/02/28 00:27:25 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/320339644137
2024/02/28 00:27:25 Could not obtain certificates:
    error: one or more domains had a problem:
[sin.mydomain.com] propagation: time limit exceeded: last error: NS eric.ns.cloudflare.com. did not return the expected TXT record [fqdn: _acme-challenge.sin.mydomain.com., value: -i2EECZPQ2U41NfVw91A1OHqThIeOevNeg58xXaf6wc]: M_R0H5ArpLUn2PJH6GmCSCz84YysbEJL3w-ExC6QXPU

The TXT entry is deleted after failure above; if I re-run your script the same behavior is observed (2) TXT entries with different values is created at cloudflare. There is no debug indicator to tell me which string is the valid one but also I don't think the second entry should exist in the first place.

[TheLinuxGuy]

I am not sure about the root cause of the multiple TXT entries for a single binary run, but wanted to add that DNS on synology was being painful for cloudflare.

I was finally able to get this working at generating certs by passing this option in the env file

LEGO_OPTIONS=(--dns.resolvers "1.1.1.1")

[JessThrysoee]

Is 192.168.20.1:53 a local DNS resolver cache, like dnsmasq? If so, then it probably cached the first NXDOMAIN response (negative caching) and keeps returning that without forwarding.

By setting --dns.resolvers you are bypassing your local cache and the recursive queries to 1.1.1.1 will at first receive NXDOMAIN but eventually resolve the _acme-challenge TXT.

If you use dnsmasq; it has a no-negcache flag, but I can't know if that is appropriate for your setup.

[TheLinuxGuy]

Is 192.168.20.1:53 a local DNS resolver cache, like dnsmasq? If so, then it probably cached the first NXDOMAIN response (negative caching) and keeps returning that without forwarding.

Yes, its a firewalla.com router device. Bypassing the DNS resolver from DHCP allowed the script to run and I guess "resolved" the issue - although its weird that 2 API calls to Cloudflare to create 2 distinct TXT records were executed in a single run.

I don't think there is much value in debugging further as I seem to be an edge-case. Going to close this - thanks!