PowerDNS / lightningstream

Lightning Stream syncs LMDB databases through S3 buckets between multiple servers, including PowerDNS Authoritative server 4.8+ LMDBs
https://doc.powerdns.com/lightningstream/
MIT License
25 stars 15 forks source link

Experimental pdns-v5-fix-duplicate-domains command #50

Closed wojas closed 1 year ago

wojas commented 1 year ago

The experimental pdns-v5-fix-duplicate-domains command can be used to fix duplicate domain entries in pdns auth 4.8 schemaversion 5 LMDBs.

If this is ever merged, it will likely be removed again once PDNS Auth has an internal mechanism to resolve this.

Example run:

$ lightningstream --log-level debug -c pdns-native.yaml experimental pdns-v5-fix-duplicate-domains -d main --dangerous-do-rename
DEBU[0000] Running                                       version=dev
DEBU[0000] Scan                                          domain=dup.example domain_id=428582697
DEBU[0000] Scan                                          domain=dup.example domain_id=579235200
ERRO[0000] Duplicate domain entry!                       domain=dup.example domain_id=579235200 header_ts="2023-03-31 17:57:02.07626496 +0800 CST" keeping_oldest_id=428582697 prev_domain_id=428582697 prev_header_ts="2023-03-31 17:56:59.632763904 +0800 CST" renaming_newest_id=579235200
WARN[0000] PATCHING                                      domain=dup.example domain_id=579235200 flags=1 key="..example.dup.\".m. [00 0c 65 78 61 6d 70 6c 65 00 64 75 70 00 22 86 6d 80]" val=".Q|.N..P................ [17 51 7c ec 4e 99 dc 50 00 00 00 00 00 00 00 0d 00 01 00 00 00 00 00 00] (2023-03-31T11:18:10.358738Z)"
WARN[0000] PATCHING                                      domain=dup.example.dup-579235200.invalid domain_id=579235200 flags=0 key=".\"invalid.dup-579235200.example.dup.\".m. [00 22 69 6e 76 61 6c 69 64 00 64 75 70 2d 35 37 39 32 33 35 32 30 30 00 65 78 61 6d 70 6c 65 00 64 75 70 00 22 86 6d 80]" val=".Q|.N.YP................ [17 51 7c ec 4e 9a 59 50 00 00 00 00 00 00 00 0d 00 00 00 00 00 00 00 00] (2023-03-31T11:18:10.35877Z)"
DEBU[0000] Scan                                          domain=dup.example.dup-579235200.invalid domain_id=579235200
INFO[0000] Done

The following zones need to be removed with `pdnsutil delete-zone ZONE`:

- dup.example.dup-579235200.invalid

Note that these will NOT show up in list-all-zones. Removing the zones will also not remove it from this list.

Additionally, this switches the Linux binary builds from Ubuntu 22.04 to 20.04 for greater compatibility with other distributions. This is needed, because we use CGo for the LMDB bindings.