Caddy: Various issues updating Dynamic DNS entry for a subdomain of a wildcard

no-usernames-left commented 3 months ago

Important notices

[X] I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
[X] I have searched the existing issues, open and closed, and I'm convinced that mine is new.
[X] The title contains the plugin to which this issue belongs

Describe the bug With a wildcard domain and two subdomains as configured here, I wish to have Dynamic DNS update only one of the subdomains, fw.DOMAIN.TLD, which is an A record. (The other subdomain is a CNAME pointing to fw, and therefore Dynamic DNS is only enabled for fw.)

Caddy is able to fetch a certificate using a DNS-01 challenge, so I know the Gandi token is good. However, the A record is not being updated.

Expected behavior No error message and a single A record reflecting the tracked interface's IP.

Actual behavior There was already an A record for fw in the DOMAIN.TLD zone with TTL 300 (the shortest TTL supported by Gandi). When the IP of the tracked interface matched what was already in DNS, the log output below was generated and no changes were made in Gandi DNS.

Most recent output at the top, with the first line being the error in question:

2024-06-07T21:13:13 Informational   caddy   "info","ts":"2024-06-07T21:13:13Z","logger":"dynamic_dns","msg":"domain not found in DNS","domain":"fw.DOMAIN.TLD"}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"tls","msg":"finished cleaning storage units"}
2024-06-07T21:13:12 Warning caddy   "warn","ts":"2024-06-07T21:13:12Z","logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/var/db/caddy/data/caddy","instance":"7dd70669-0cc4-4dfb-bf2b-ab10deedd5b9","try_again":"2024-06-08T21:13:12Z","try_again_in":86399.999999549}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","msg":"serving initial configuration"}   
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","msg":"autosaved config (load with --resume flag)","file":"/var/db/caddy/config/caddy/autosave.json"}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"http","msg":"enabling automatic TLS certificate management","domains":["*.DOMAIN.TLD"]}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x86c65a300"}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"admin","msg":"admin endpoint started","address":"unix//var/run/caddy/caddy.sock","enforce_origin":false,"origins":["","//127.0.0.1","//::1"]}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"admin.api","msg":"shutdown complete","exit_code":0}
2024-06-07T21:13:12 Informational   caddy   "info","ts":"2024-06-07T21:13:12Z","logger":"admin","msg":"stopped previous server","address":"unix//var/run/caddy/caddy.sock"}
2024-06-07T21:12:59 Informational   caddy   "info","ts":"2024-06-07T21:12:59Z","logger":"http","msg":"servers shutting down with eternal grace period"}
2024-06-07T21:12:59 Warning caddy   "warn","ts":"2024-06-07T21:12:59Z","logger":"admin.api","msg":"exiting; byeee!! 👋"}
2024-06-07T21:12:59 Informational   caddy   "info","ts":"2024-06-07T21:12:59Z","logger":"admin.api","msg":"received request","method":"POST","host":"127.0.0.1","uri":"/stop","remote_ip":"","remote_port":"","headers":{"Accept-Encoding":["gzip"],"Content-Length":["0"],"User-Agent":["Go-http-client/1.1"]}}

However, while troubleshooting this issue I manually set the IP of the A record to 0.0.0.0, restarted Caddy, and:

The TTL of that 0.0.0.0 record was changed from 300 to 3600 (matching the 1-hour TTL configured in the web GUI), but the IP was not changed; and
a new A record (yes, duplicate label of fw) was created with TTL 3600 and the correct IP, resulting in both records being returned in queries. This is broken.

Here's the log output in that case, throwing the same domain not found in DNS error as above, but with two extra records above it which I was glad to see:

2024-06-07T21:25:35 Informational   caddy   "info","ts":"2024-06-07T21:25:35Z","logger":"dynamic_dns","msg":"finished updating DNS","current_ips":["IP.ADD.RE.SS"]}
2024-06-07T21:25:35 Informational   caddy   "info","ts":"2024-06-07T21:25:35Z","logger":"dynamic_dns","msg":"updating DNS record","zone":"DOMAIN.TLD","type":"A","name":"fw","value":"IP.ADD.RE.SS","ttl":3600}
2024-06-07T21:25:35 Informational   caddy   "info","ts":"2024-06-07T21:25:35Z","logger":"dynamic_dns","msg":"domain not found in DNS","domain":"fw.DOMAIN.TLD"}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"tls","msg":"finished cleaning storage units"}
2024-06-07T21:25:34 Warning caddy   "warn","ts":"2024-06-07T21:25:34Z","logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/var/db/caddy/data/caddy","instance":"7dd70669-0cc4-4dfb-bf2b-ab10deedd5b9","try_again":"2024-06-08T21:25:34Z","try_again_in":86399.999999349}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","msg":"serving initial configuration"}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","msg":"autosaved config (load with --resume flag)","file":"/var/db/caddy/config/caddy/autosave.json"}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"http","msg":"enabling automatic TLS certificate management","domains":["*.DOMAIN.TLD"]}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x86c0d1580"}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"admin","msg":"admin endpoint started","address":"unix//var/run/caddy/caddy.sock","enforce_origin":false,"origins":["","//127.0.0.1","//::1"]}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"admin.api","msg":"shutdown complete","exit_code":0}
2024-06-07T21:25:34 Informational   caddy   "info","ts":"2024-06-07T21:25:34Z","logger":"admin","msg":"stopped previous server","address":"unix//var/run/caddy/caddy.sock"}
2024-06-07T21:25:21 Informational   caddy   "info","ts":"2024-06-07T21:25:21Z","logger":"http","msg":"servers shutting down with eternal grace period"}
2024-06-07T21:25:21 Warning caddy   "warn","ts":"2024-06-07T21:25:21Z","logger":"admin.api","msg":"exiting; byeee!! 👋"}
2024-06-07T21:25:21 Informational   caddy   "info","ts":"2024-06-07T21:25:21Z","logger":"admin.api","msg":"received request","method":"POST","host":"127.0.0.1","uri":"/stop","remote_ip":"","remote_port":"","headers":{"Accept-Encoding":["gzip"],"Content-Length":["0"],"User-Agent":["Go-http-client/1.1"]}}

Environment OPNsense Business 24.4_8 (amd64) caddy-custom 2.7.6.5.0.3.5.516 (wow)_ os-caddy 1.5.4_1

Monviech commented 3 months ago

I have to refer you upstream for this. The dynamic DNS configuration that my template generates should be valid for most providers.

The issue is probably in the compiled module using the API in unexpected ways.

If the (potential) issue is fixed there we can downstream it into a new caddy binary.

https://github.com/caddy-dns/gandi

Monviech commented 3 months ago

Maybe implementing this could potentially fix it, but unsure since it could be still an upstream issue too.:

https://github.com/mholt/caddy-dynamicdns/blob/012a1d4347472eaf4b78826b86c8f35bda919f72/dynamicdns.go#L56-L58

no-usernames-left commented 3 months ago

Maybe implementing this could potentially fix it

At the very minimum there is a bug where the 0.0.0.0 record had its TTL updated and a new record was also created with the correct IP, which is broken no matter how you look at it. So I am not confident on this; it looks like the update-vs-create logic in the Gandi module is buggy.

opnsense / plugins

Caddy: Various issues updating Dynamic DNS entry for a subdomain of a wildcard #4036