DNS Entries are not overwritten, they are appended

mietzen commented 10 months ago

I'm using dynamic_dns with provider porkbun the A records are created, but instead of overwriting the old IP with with the new one, the old entry is left untouched and a new one is created.

I first opened a ticket in dynamic_dns: https://github.com/mholt/caddy-dynamicdns/issues/49

But I actually think the problem lays down here in, potentially in: https://github.com/libdns/porkbun/blob/27b0dc30eeffb0020ba96d397a8b806ff36b6df8/provider.go#L92

DNS-Entries

dynamic_dns seems to find the records, but doesn't associate them with the ones it should update.

2023/09/02 09:24:58 DEBUG   dynamic_dns found DNS record    {"type": "A", "name": "wireguard.home.mietzen.xyz", "zone": "mietzen.xyz", "value": "xxx.xxx.xxx.123"}
...
2023/09/02 09:24:58 INFO    dynamic_dns domain not found in DNS {"domain": "wireguard.home.mietzen.xyz"}
...
2023/09/02 09:24:58 DEBUG   dynamic_dns looked up current IPs from DNS  {"lastIPs": {"home-assistant.home.mietzen.xyz":{"A":[""],"AAAA":[""]},"vaultwarden.home.mietzen.xyz":{"A":[""],"AAAA":[""]},"wireguard.home.mietzen.xyz":{"A":[""],"AAAA":[""]}}}
...
2023/09/02 09:24:59 INFO    dynamic_dns updating DNS record {"zone": "mietzen.xyz", "type": "A", "name": "wireguard.home", "value": "xxx.xxx.xxx.172", "ttl": 3600}

My best guess is that dynamic_dns expects the zone to not be included in the name.

Edit:

As written here: https://porkbun.com/api/json/v3/documentation#DNS%20Retrieve%20Records%20by%20Domain%20or%20ID

The response name always include the zone:

        {
            "id": "106926659",
            "name": "www.borseth.ink",
            "type": "A",
            "content": "1.1.1.1",
            "ttl": "600",
            "prio": "0",
            "notes": ""
        }

And as far as I understand isn't trimmed in GetRecords.

mietzen commented 10 months ago

Thanks for the quick response, unfortunately that did fix this Issue:

Modules:

./caddy list-modules --versions
admin.api.load v2.7.4
admin.api.metrics v2.7.4
admin.api.pki v2.7.4
admin.api.reverse_proxy v2.7.4
caddy.adapters.caddyfile v2.7.4
caddy.config_loaders.http v2.7.4
caddy.listeners.http_redirect v2.7.4
caddy.listeners.proxy_protocol v2.7.4
caddy.listeners.tls v2.7.4
caddy.logging.encoders.console v2.7.4
caddy.logging.encoders.filter v2.7.4
caddy.logging.encoders.filter.cookie v2.7.4
caddy.logging.encoders.filter.delete v2.7.4
caddy.logging.encoders.filter.hash v2.7.4
caddy.logging.encoders.filter.ip_mask v2.7.4
caddy.logging.encoders.filter.query v2.7.4
caddy.logging.encoders.filter.regexp v2.7.4
caddy.logging.encoders.filter.rename v2.7.4
caddy.logging.encoders.filter.replace v2.7.4
caddy.logging.encoders.json v2.7.4
caddy.logging.writers.discard v2.7.4
caddy.logging.writers.file v2.7.4
caddy.logging.writers.net v2.7.4
caddy.logging.writers.stderr v2.7.4
caddy.logging.writers.stdout v2.7.4
caddy.storage.file_system v2.7.4
events v2.7.4
http v2.7.4
http.authentication.hashes.bcrypt v2.7.4
http.authentication.hashes.scrypt v2.7.4
http.authentication.providers.http_basic v2.7.4
http.encoders.gzip v2.7.4
http.encoders.zstd v2.7.4
http.handlers.acme_server v2.7.4
http.handlers.authentication v2.7.4
http.handlers.copy_response v2.7.4
http.handlers.copy_response_headers v2.7.4
http.handlers.encode v2.7.4
http.handlers.error v2.7.4
http.handlers.file_server v2.7.4
http.handlers.headers v2.7.4
http.handlers.invoke v2.7.4
http.handlers.map v2.7.4
http.handlers.metrics v2.7.4
http.handlers.push v2.7.4
http.handlers.request_body v2.7.4
http.handlers.reverse_proxy v2.7.4
http.handlers.rewrite v2.7.4
http.handlers.static_response v2.7.4
http.handlers.subroute v2.7.4
http.handlers.templates v2.7.4
http.handlers.tracing v2.7.4
http.handlers.vars v2.7.4
http.ip_sources.static v2.7.4
http.matchers.client_ip v2.7.4
http.matchers.expression v2.7.4
http.matchers.file v2.7.4
http.matchers.header v2.7.4
http.matchers.header_regexp v2.7.4
http.matchers.host v2.7.4
http.matchers.method v2.7.4
http.matchers.not v2.7.4
http.matchers.path v2.7.4
http.matchers.path_regexp v2.7.4
http.matchers.protocol v2.7.4
http.matchers.query v2.7.4
http.matchers.remote_ip v2.7.4
http.matchers.vars v2.7.4
http.matchers.vars_regexp v2.7.4
http.precompressed.br v2.7.4
http.precompressed.gzip v2.7.4
http.precompressed.zstd v2.7.4
http.reverse_proxy.selection_policies.client_ip_hash v2.7.4
http.reverse_proxy.selection_policies.cookie v2.7.4
http.reverse_proxy.selection_policies.first v2.7.4
http.reverse_proxy.selection_policies.header v2.7.4
http.reverse_proxy.selection_policies.ip_hash v2.7.4
http.reverse_proxy.selection_policies.least_conn v2.7.4
http.reverse_proxy.selection_policies.query v2.7.4
http.reverse_proxy.selection_policies.random v2.7.4
http.reverse_proxy.selection_policies.random_choose v2.7.4
http.reverse_proxy.selection_policies.round_robin v2.7.4
http.reverse_proxy.selection_policies.uri_hash v2.7.4
http.reverse_proxy.selection_policies.weighted_round_robin v2.7.4
http.reverse_proxy.transport.fastcgi v2.7.4
http.reverse_proxy.transport.http v2.7.4
http.reverse_proxy.upstreams.a v2.7.4
http.reverse_proxy.upstreams.multi v2.7.4
http.reverse_proxy.upstreams.srv v2.7.4
pki v2.7.4
tls v2.7.4
tls.certificates.automate v2.7.4
tls.certificates.load_files v2.7.4
tls.certificates.load_folders v2.7.4
tls.certificates.load_pem v2.7.4
tls.certificates.load_storage v2.7.4
tls.client_auth.leaf v2.7.4
tls.get_certificate.http v2.7.4
tls.get_certificate.tailscale v2.7.4
tls.handshake_match.remote_ip v2.7.4
tls.handshake_match.sni v2.7.4
tls.issuance.acme v2.7.4
tls.issuance.internal v2.7.4
tls.issuance.zerossl v2.7.4
tls.stek.distributed v2.7.4
tls.stek.standard v2.7.4

  Standard modules: 106

dns.providers.porkbun v0.1.3
dynamic_dns v0.0.0-20230706012752-3cdd858980a4
dynamic_dns.ip_sources.command v0.0.0-20230424194406-b23dd5fa6436
dynamic_dns.ip_sources.interface v0.0.0-20230706012752-3cdd858980a4
dynamic_dns.ip_sources.simple_http v0.0.0-20230706012752-3cdd858980a4
dynamic_dns.ip_sources.upnp v0.0.0-20230706012752-3cdd858980a4

  Non-standard modules: 6

  Unknown modules: 0

Debug log:

2023/09/05 10:29:10 DEBUG   dynamic_dns found DNS record    {"type": "A", "name": "wireguard.home.mietzen.xyz.", "zone": "mietzen.xyz", "value": "xxx.xxx.xxx.123"}
2023/09/05 10:29:10 DEBUG   dynamic_dns found DNS record    {"type": "A", "name": "home-assistant.home.mietzen.xyz.", "zone": "mietzen.xyz", "value": "xxx.xxx.xxx.123"}
2023/09/05 10:29:10 DEBUG   dynamic_dns found DNS record    {"type": "A", "name": "vaultwarden.home.mietzen.xyz.", "zone": "mietzen.xyz", "value": "xxx.xxx.xxx.123"}
2023/09/05 10:29:10 INFO    dynamic_dns domain not found in DNS {"domain": "wireguard.home.mietzen.xyz"}
2023/09/05 10:29:10 INFO    dynamic_dns domain not found in DNS {"domain": "wireguard.home.mietzen.xyz"}
2023/09/05 10:29:10 INFO    dynamic_dns domain not found in DNS {"domain": "home-assistant.home.mietzen.xyz"}
2023/09/05 10:29:10 INFO    dynamic_dns domain not found in DNS {"domain": "home-assistant.home.mietzen.xyz"}
2023/09/05 10:29:10 INFO    dynamic_dns domain not found in DNS {"domain": "vaultwarden.home.mietzen.xyz"}
2023/09/05 10:29:10 INFO    dynamic_dns domain not found in DNS {"domain": "vaultwarden.home.mietzen.xyz"}
2023/09/05 10:29:10 DEBUG   dynamic_dns looked up current IPs from DNS  {"lastIPs": {"home-assistant.home.mietzen.xyz":{"A":[""],"AAAA":[""]},"vaultwarden.home.mietzen.xyz":{"A":[""],"AAAA":[""]},"wireguard.home.mietzen.xyz":{"A":[""],"AAAA":[""]}}}
2023/09/05 10:29:10 DEBUG   dynamic_dns.ip_sources.command  running command {"command": "/opt/CaddyV2/fritzbox_ext_ip", "args": ["192.168.178.1"], "dir": "", "timeout": 30000000000}
2023/09/05 10:29:10 DEBUG   dynamic_dns.ip_sources.command  parsed ip succesfull    {"command": "/opt/CaddyV2/fritzbox_ext_ip", "args": ["192.168.178.1"], "stdout": "xxx.xxx.xxx.120\n", "ip": "xxx.xxx.xxx.120"}
2023/09/05 10:29:10 INFO    dynamic_dns updating DNS record {"zone": "mietzen.xyz", "type": "A", "name": "wireguard.home", "value": "xxx.xxx.xxx.120", "ttl": 3600}
2023/09/05 10:29:10 INFO    dynamic_dns updating DNS record {"zone": "mietzen.xyz", "type": "A", "name": "home-assistant.home", "value": "xxx.xxx.xxx.120", "ttl": 3600}
2023/09/05 10:29:10 INFO    dynamic_dns updating DNS record {"zone": "mietzen.xyz", "type": "A", "name": "vaultwarden.home", "value": "xxx.xxx.xxx.120", "ttl": 3600}
2023/09/05 10:29:13 INFO    dynamic_dns finished updating DNS   {"current_ips": ["xxx.xxx.xxx.120"]}

Stdout:

./caddy run --envfile /opt/CaddyV2/.env
2023/09/05 08:33:56.994 INFO    using adjacent Caddyfile
2023/09/05 08:33:57.104 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2023/09/05 08:33:57.108 INFO    tls.cache.maintenance   started background certificate maintenance  {"cache": "0x8706c3800"}
2023/09/05 08:33:57.109 INFO    http.auto_https server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
2023/09/05 08:33:57.109 INFO    http.auto_https enabling automatic HTTP->HTTPS redirects    {"server_name": "srv0"}
2023/09/05 08:33:57.136 INFO    http    enabling HTTP/3 listener    {"addr": ":443"}
2023/09/05 08:33:57.138 INFO    tls cleaning storage unit   {"description": "FileStorage:/opt/CaddyV2/data"}
2023/09/05 08:33:57.142 INFO    http.log    server running  {"name": "srv0", "protocols": ["h1", "h2", "h3"]}
https://porkbun.com/api/json/v3/dns/retrieve/mietzen.xyz
2023/09/05 08:33:57.160 INFO    tls finished cleaning storage units
2023/09/05 08:33:57.161 INFO    http.log    server running  {"name": "remaining_auto_https_redirects", "protocols": ["h1", "h2", "h3"]}
2023/09/05 08:33:57.161 INFO    http    enabling automatic TLS certificate management   {"domains": ["home.mietzen.xyz", "*.home.mietzen.xyz"]}
2023/09/05 08:33:57.189 INFO    autosaved config (load with --resume flag)  {"file": "/root/.config/caddy/autosave.json"}
2023/09/05 08:33:57.189 INFO    serving initial configuration
2023/09/05 08:33:58.499 INFO    dynamic_dns domain not found in DNS {"domain": "wireguard.home.mietzen.xyz"}
2023/09/05 08:33:58.499 INFO    dynamic_dns domain not found in DNS {"domain": "wireguard.home.mietzen.xyz"}
2023/09/05 08:33:58.499 INFO    dynamic_dns domain not found in DNS {"domain": "home-assistant.home.mietzen.xyz"}
2023/09/05 08:33:58.500 INFO    dynamic_dns domain not found in DNS {"domain": "home-assistant.home.mietzen.xyz"}
2023/09/05 08:33:58.500 INFO    dynamic_dns domain not found in DNS {"domain": "vaultwarden.home.mietzen.xyz"}
2023/09/05 08:33:58.500 INFO    dynamic_dns domain not found in DNS {"domain": "vaultwarden.home.mietzen.xyz"}
2023/09/05 08:33:58.599 INFO    dynamic_dns updating DNS record {"zone": "mietzen.xyz", "type": "A", "name": "wireguard.home", "value": "xxx.xxx.xxx.120", "ttl": 3600}
2023/09/05 08:33:58.599 INFO    dynamic_dns updating DNS record {"zone": "mietzen.xyz", "type": "A", "name": "home-assistant.home", "value": "xxx.xxx.xxx.120", "ttl": 3600}
2023/09/05 08:33:58.600 INFO    dynamic_dns updating DNS record {"zone": "mietzen.xyz", "type": "A", "name": "vaultwarden.home", "value": "xxx.xxx.xxx.120", "ttl": 3600}
https://porkbun.com/api/json/v3/dns/retrieve/mietzen.xyz
https://porkbun.com/api/json/v3/dns/create/mietzen.xyz
https://porkbun.com/api/json/v3/dns/create/mietzen.xyz
https://porkbun.com/api/json/v3/dns/create/mietzen.xyz
2023/09/05 08:34:01.699 INFO    dynamic_dns finished updating DNS   {"current_ips": ["xxx.xxx.xxx.120"]}
2023/09/05 08:34:10.325 INFO    shutting down   {"signal": "SIGINT"}
2023/09/05 08:34:10.325 WARN    exiting; byeee!! 👋  {"signal": "SIGINT"}
2023/09/05 08:34:10.326 INFO    http    servers shutting down with eternal grace period
2023/09/05 08:34:10.333 INFO    admin   stopped previous server {"address": "localhost:2019"}
2023/09/05 08:34:10.333 INFO    shutdown complete   {"signal": "SIGINT", "exit_code": 0}

Niallfitzy1 commented 10 months ago

I'll try to get another look at this soon

Thanks for the detailed logs

Niallfitzy1 commented 10 months ago

@mietzen I've got a branch up that hopefully fixes this. I want to test it more before rolling it out but if you get the chance let me know if it solves the issue for you.

You can pull it in by specifying the version of caddy-dns/porkbun to @bump-libdns-again like this xcaddy build --with github.com/caddy-dns/porkbun@bump-libdns-again

mietzen commented 10 months ago

Just build caddy and tested it porkbun@bump-libdns-again fixes this Issue 👍

Niallfitzy1 commented 10 months ago

Thanks for testing, the fix is fully released now

mietzen commented 10 months ago

Have you changed anything else? Today I updated my caddy binary and it only updates one domain and then throws a error:

2023/09/07 14:21:53.141 INFO    dynamic_dns domain not found in DNS {"domain": "home-assistant.home"}
2023/09/07 14:21:53.216 INFO    dynamic_dns different IP address    {"new_ip": "xxx.xxx.xxx.178", "old_ips": ["xxx.xxx.xxx.178", "<nil>"]}
2023/09/07 14:21:53.216 INFO    dynamic_dns updating DNS record {"zone": "mietzen.xyz", "type": "A", "name": "home-assistant.home", "value": "xxx.xxx.xxx.178", "ttl": 3600}
2023/09/07 14:21:54.432 ERROR   dynamic_dns failed setting DNS record(s) with new IP address(es)    {"zone": "mietzen.xyz", "error": "Invalid http response status, {\"status\":\"ERROR\",\"message\":\"Edit error: We were unable to edit the DNS record.\"}"}

"Fresh" caddy:

dns.providers.ddnss v0.0.0-20221206165031-7f65108b0a62
dns.providers.porkbun v0.1.4
dynamic_dns v0.0.0-20230403023955-e774c7b03d98
dynamic_dns.ip_sources.command v0.0.0-20230424194406-b23dd5fa6436
dynamic_dns.ip_sources.simple_http v0.0.0-20230403023955-e774c7b03d98
dynamic_dns.ip_sources.upnp v0.0.0-20230403023955-e774c7b03d98

  Non-standard modules: 6

  Unknown modules: 0

Caddy with bugfixes:

dns.providers.porkbun v0.1.4-0.20230906001019-b3e67a78f7f7
dynamic_dns v0.0.0-20230706012752-3cdd858980a4
dynamic_dns.ip_sources.command v0.0.0-20230424194406-b23dd5fa6436
dynamic_dns.ip_sources.interface v0.0.0-20230706012752-3cdd858980a4
dynamic_dns.ip_sources.simple_http v0.0.0-20230706012752-3cdd858980a4
dynamic_dns.ip_sources.upnp v0.0.0-20230706012752-3cdd858980a4

  Non-standard modules: 6

  Unknown modules: 0

Yesterdays bugfix version still works fine.

Niallfitzy1 commented 7 months ago

Sorry for the delay getting back My raspberry pi died a while ago. Due to that I've not been running caddy anymore & so procrastinated on this quite a bit

I didn't change anything when publishing the new version, just merged the PRs & pushed the release tags. Not sure why there would be any difference. Did you ever try again with the released version?

If you manage to come up with a fix I'm happy to review, otherwise I'll try take a look when I get another block of time off

libdns / porkbun