Closed james-stevens closed 1 year ago
james-stevens
commented
2 years ago OK - so looks like its my server - not sure why, but spun up hsd on my dev machine (which hadn't been sync'd in months), left it to sync, then ran the AXFR and it worked -> 7,470,288 rows & both SOA records.
I had logging set to debug on my dev, but info on production
But my production server has 18Gb RAM - so shouldn't be that - although bind can take a ton due to fragmentation.
james-stevens
commented
2 years ago Beginning to think it may be bind that's the problem !
I know dot-EU use bind to dynamically sign, and their zone is pretty big
Also, looks like it hadn't sync'd cos 7.4M rows is a bit low, production is over 10M now & still going - but the dig axfr has got a lot further after I closed down bind
james-stevens
commented
2 years ago nah - something is def odd in hsd, the last record seems to be some kind of large binary object of some sort - like its put 100s of RRs into one record
hasroot:/opt/rom/tmp# dig @127.0.0.9 . axfr > /tmp/hds-root
hasroot:/opt/rom/tmp# tail !$
tail /tmp/hds-root
00 01 00 00 54 60 00 04 2c e7 06 b7 03 6e 73 32 ....T`..,....ns2
0a 6c 69 6e 67 61 73 77 61 6d 79 00 00 01 00 01 .lingaswamy.....
00 00 54 60 00 04 36 d6 88 f6 09 74 68 65 6d 6f ..T`..6....themo
72 67 75 65 00 00 02 00 01 00 00 54 60 00 0f 03 rgue.......T`...
6e 73 31 09 74 68 65 6d 6f 72 67 75 65 00 09 74 ns1.themorgue..t
68 65 6d 6f 72 67 75 65 00 00 02 00 01 00 00 54 hemorgue.......T
60 00 0f 03 6e 73 32 09 74 68 65 6d 6f 72 67 75 `...ns2.themorgu
65 00 09 74 68 65 6d 6f 72 67 75 65 00 00 2b 00 e..themorgue..+.
01 00 00 54 60 00 18 f8 d4 0d 01 f8 c0 db 58 75 ...T`.........Xu
06 9c 16 62 a0 00 bf 1b e3 75 00 6c 89 3e 81 ...b.....u.l.>.The output from dig ... axfr should be only text representation
james-stevens
commented
2 years ago This is how the text ends & the binary start - my guess would be somebody has managed to get some garbage data into the blockchain.
ns1.cabudol. 21600 IN A 44.231.6.183
iebq. 21600 IN NS ns1.iebq.
iebq. 21600 IN NS ns2.iebq.
ns1.iebq. 21600 IN A 44.231.6.183
ns2.iebq. 21600 IN A 54.214.136.246
maxonus. 21600 IN TXT "Contact: o81@protonmail.com"
barrstein. 21600 IN NS ns1.barrstein.
ns1.barrstein. 21600 IN A 44.231.6.183
;; Got bad packet: extra input data
55327 bytes
14 53 80 00 00 01 07 54 00 00 00 00 00 00 fc 00 .S.....T........
01 11 78 6e 2d 2d 6d 6e 38 68 65 61 66 36 33 6e ..xn--mn8heaf63n
62 61 62 00 00 02 00 01 00 00 54 60 00 06 03 6e bab.......T`...n
73 31 c0 11 c0 2e 00 01 00 01 00 00 54 60 00 04 s1..........T`..
2c e7 06 b7 08 73 69 6e 67 73 68 6f 70 00 00 02 ,....singshop...
00 01 00 00 54 60 00 06 03 6e 73 31 c0 44 c0 44 ....T`...ns1.D.D
00 02 00 01 00 00 54 60 00 06 03 6e 73 32 c0 44 ......T`...ns2.D
c0 58 00 01 00 01 00 00 54 60 00 04 2c e7 06 b7 .X......T`..,...
c0 6a 00 01 00 01 00 00 54 60 00 04 36 d6 88 f6 .j......T`..6...Still no closer to finding out what bind / dig are moaning about with your AXFR.
I tried different packet sizes, in case that was the issue - DNS compression only works up to packet sizes of 16384, so I thought using 64K packet sizes might cause the problem - but not.
No matter the packet size, it fails at about the same place in the zone transfer - you get about the same amount of valid data before it fails with 50K, 5K & 1.5K packets.
I've also taken the hex it outputs, which I'm assuming is meant to be the "bad packet", & pushed that through my own DNS packet parser, and my parser was perfectly happy with it. So what ISC seems to be calling a "bad packet" doesn't bother my parser.
With 5K and 1.5K packet sizes the AXFR terminates at almost exactly the same place
5K end of packet
...
mgar. 21600 IN NS ns1.mgar.
ns1.mgar. 21600 IN A 207.180.198.58
92299. 21600 IN NS ns1.92299.
92299. 21600 IN NS ns2.92299.
ns1.92299. 21600 IN A 44.231.6.183
ns2.92299. 21600 IN A 54.214.136.246
nywr. 21600 IN NS ns1.nywr.1.5K end of packet
mgar. 21600 IN NS ns1.mgar.
ns1.mgar. 21600 IN A 207.180.198.58
92299. 21600 IN NS ns1.92299.
92299. 21600 IN NS ns2.92299.
ns1.92299. 21600 IN A 44.231.6.183
ns2.92299. 21600 IN A 54.214.136.246
nywr. 21600 IN NS ns1.nywr.
nywr. 21600 IN NS ns2.nywr.
ns1.nywr. 21600 IN A 44.231.6.183
ns2.nywr. 21600 IN A 54.214.136.246This is the entire decoding of the last packet when I have 1.5K packet sizes set (642 bytes) - I only get about 3.6M names, so this is far from being the last packet - the Handshake ROOT zone is much bigger.
My only immediate comment would be you shouldn't have QR set, or opcode: QUERY, as this is a response - but if that was the problem the AXFR sould have terminated a long time before :)
ANSWER: 30 is correct, for this packet.
;;
;; HEADER: opcode: QUERY, status: NOERROR, id: 9562
;; flags: qr; QUERY: 1, ANSWER: 30, AUTHORITY: 0, ADDITIONAL: 0
QUESTION SECTION:
. IN AXFR
;; ANSWER SECTION:
adamsimms. 21600 IN NS ns2.adamsimms.
ns1.adamsimms. 21600 IN A 44.231.6.183
ns2.adamsimms. 21600 IN A 54.214.136.246
pirireis. 21600 IN NS ns1.pirireis.
pirireis. 21600 IN NS ns2.pirireis.
ns1.pirireis. 21600 IN A 44.231.6.183
ns2.pirireis. 21600 IN A 54.214.136.246
xn--emma-3o1is3mn66b. 21600 IN NS ns1.xn--emma-3o1is3mn66b.
xn--emma-3o1is3mn66b. 21600 IN NS ns2.xn--emma-3o1is3mn66b.
ns1.xn--emma-3o1is3mn66b.21600 IN A 44.231.6.183
ns2.xn--emma-3o1is3mn66b.21600 IN A 54.214.136.246
gopot. 21600 IN NS ns1.gopot.
gopot. 21600 IN NS ns2.gopot.
ns1.gopot. 21600 IN A 44.231.6.183
ns2.gopot. 21600 IN A 54.214.136.246
ceskedrahy. 21600 IN NS ns1.ceskedrahy.
ceskedrahy. 21600 IN NS ns2.ceskedrahy.
ns1.ceskedrahy. 21600 IN A 44.231.6.183
ns2.ceskedrahy. 21600 IN A 54.214.136.246
mgar. 21600 IN DS 9264 4 2 8E4A3930A80F2FDAFEDA9782F69AB2EB4538D7F5CA23D8B4E0F17173 AA69899B7D0D099B46D08D52E6BA7F7A3B3C5AD0
mgar. 21600 IN NS ns1.mgar.
ns1.mgar. 21600 IN A 207.180.198.58
92299. 21600 IN NS ns1.92299.
92299. 21600 IN NS ns2.92299.
ns1.92299. 21600 IN A 44.231.6.183
ns2.92299. 21600 IN A 54.214.136.246
nywr. 21600 IN NS ns1.nywr.
nywr. 21600 IN NS ns2.nywr.
ns1.nywr. 21600 IN A 44.231.6.183
ns2.nywr. 21600 IN A 54.214.136.246Each TCP packet starts with a 2 byte unsigned int as the length of the DNS data - I'm not sure if dig is reporting this value or not. The error report starts like this
;; Got bad packet: extra input data
642 bytes
25 5a 80 00 00 01 00 1e 00 00 00 00 00 00 fc 00 %Z..............If that 642 bytes is the length reported by TCP, then it does match the number of bytes in the hex dump.
But its not clear if the 642 is just telling me the length of the hex dump, or is the TCP packet length.
When my parser runs on the hex dump, it finishes at 642, so is happy with the packet.
james-stevens
commented
1 year ago I will leave it for now & run it again in a few days to see if it still terminates at the same name.
I also have my own AXFR reader, so I will try that.
james-stevens
commented
1 year ago There is one thing that stands out in that data - the DS for mgar is a little odd - alg=4is Reserved - but I would have expected a slightly more useful error message, if this was the problem - but you never know with bind - if may have run a validation on the data, which failed, and this was the only way it had of expressing that??
https://www.iana.org/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xhtml
https://www.rfc-editor.org/rfc/rfc6725.html <- Section 2.1 changes 4 to Reserved
======================
Just checked - there are NO others with alg=4, so could be that
james-stevens
commented
1 year ago OK - after some code changes, my AXFR works fine with this plug-in - so I'm even more thinking its the alg=4 - the only way to be sure would be with the code for bind. It would prob be easy to disable the check, but they'd never take the update, so it would mean maintaining a fork, which is a pain - unless the record was filtered out at the hsd end.
The main code change I had to make was to my NS sanity checker - cos it wouldn't accept @, : and ! as characters in an NS name :rofl: ... yes, one of the NS is ns1.hi! - as hi! doesn't exist as a TLD that's not going to resolve.
james-stevens
commented
1 year ago I'm not going to call this a "fix" or "work-around" & def not going to raise a PR, but it does solve the problem & mean I can once against AXFR into bind.
This reminds me of these changes, that come with the comment "we'll fix this properly later", but it's been like that for at least 10 yrs.
--- /tmp/axfr.js
+++ axfr.js
@@ -90,6 +90,9 @@
}
pickRRs(name, hnsZone, mergeDB) {
+
+ if (name == "mgar.") return [];
+
const collision = mergeDB.nsecChain.has(name);
if (!collision) {
return hnsZone;I'm assuming that if there is any kind of DS record validation library code in hsd somewhere, that it would give this record the OK anyway - obv, I'd be of the opinion that must be some DS validation somewhere that needs correcting to validate the algorithm parameter as per the RFC/IANA (as above) - so the user is never allowed to put this into the ROOT in the first place.
I still think binds behavior could be better but, in a validating resolver, that TLD should never resolve. Because it has a DS record, no data can be returned that hasn't been validated, but nothing can be validated with alg=4, so simply removing it from the ROOT zone (as per this patch) is actually pretty much the same behaviour.
A failed validation would probably give a SERVFAIL not an NXDOMAIN, but either way, what the user sees would be much the same.
james-stevens
commented
1 year ago It's a "fix" of sorts - better than the previous one :)
I've tested it a bit, but running a full AXFR takes about 1 hr 45 right now :)
$ diff /tmp/axfr.js axfr.js
--- /tmp/axfr.js
+++ axfr.js
@@ -32,6 +32,8 @@
const {
Message,
types,
+ algs,
+ algsByVal,
codes
} = wire;
@@ -205,6 +209,13 @@
continue;
for (const rr of zone) {
+
+ if (rr.type == types.DS) {
+ const rd = rr.data;
+ if (!(rd.algorithm in algsByVal)) continue;
+ if (rd.algorithm == algs.ECC) continue;
+ }
+
if (rr.type === types.RRSIG)
continue;
Looks like that fixes dig, but not named - so there's prob another record it doesn't like or something
This is literally all it gives, even with logging set to debug
transfer of './IN' from 192.168.3.165#53: failed while receiving responses: unexpected end of inputI'm going to try dumping the zone using dig then run a named-checkzone on it & see if that find something
The AXFR is also transferring the TXT records, which it shouldn't do, if NS records are present - but I don't think that would break bind as its always worked that way, but I'll make a patch to fix that.
buffrr
commented
1 year ago Thanks James! I haven't had a chance to check this out yet. Were you able to figure out the issue? If not, I can take a look this weekend
james-stevens
commented
1 year ago Were you able to figure out the issue? If not, I can take a look this weekend
TL;DR - No, but I've also forked this to track a few minor changes
alg=4 - as above, but very slightly different nowNS are present, the TXTs get filtered, if a CNAME is present everything except that gets filtered. This may never happen, but named wouldn't like to see CNAME plus other records present.Filtering the TXT when NS are present is something named will do anyway - so filtering them in this plug-in just saves named from having to handle them, this includes signing them, even tho they never get used!!
https://github.com/buffrr/hsd-axfr/compare/main...james-stevens:hsd-axfr:master
(ext_util was for debugging, so isn't needed any more)
I've changed the alg=4 filter slightly so it checks that algs.ECC still exists in case they deprecate it. I did raise an issue for it, so they might.
The change in client.js was to help find the exact RR that was causing the problem, by being able to reduce the number of RRs in each TCP block.
So here's the weird results - dig @[hsd] . axfr > file worked fine after I added the first filter for alg=4, but named still couldn't axfr the zone in directly from hsd & always stopped in much the same place - but neither named nor hsd gave any meaningful message - so needs a lot more investigation.
To start the investigation, I used a tool I have, called axfr_file, that can provide an axfr from a plain text zone file & used that to axfr the zone output from dig, into named. I expected it to fail at the same place & axfr_file has a ton of debug options. However, it worked fine!!
So dig @[hsd] . axrf > file, then axfr_file [file] -> named works, but hsd->named doesn't, although (in theory) it's the exact same data, in the same order.
The only big difference I can see between axfr_file & this plug-in is that I don't use DNS compression - it's just not really worth the compute cost for a TCP stream. But I still pack multiple RRs into a single write, up to 30K bytes.
I've not tried direct axfr (from hsd to bind) since I added the second record filter. This plug-in has worked before without my filters, but it's possible somebody has added some strange record or TXT that is causing the compression in hsd to trip-up?
But, TBH, I quite like dumping to a file, then loading the zone into named using axfr_file, cos
named slave will never expire the zone & drop it.I could (should?) also extend the expiry in the SOA so its very long, but I'd still need to fix the hsd->named issue & TBH, there doesn't seem much advantage, right now - but being able to axfr over a TCP stream does make running multiple hsd and failing-over between them much easier then when it dumped direct to a file.
In some ways, I'd quite like to find out why named won't accept the zone direct from hsd. It will take a lot of time & effort, but there is an issue in using axfr_file which is that it isn't open source - its part of giant-dns.
One question I would have is - Is it possible to not use DNS compression in this plug-in?
BTW: axfr_file of the Handshake ROOT from a text file to a TCP stream & into named takes literally 5 mins, but getting it out of hsd in the first place takes nearly 2 hrs !!
james-stevens
commented
1 year ago The sheer size of the Handshake ROOT zone now makes testing a long & trying experience
named about 17Gb of RAM to store the zonenamed a few days to sign the zone (2.5 days & counting, so far - but its on a really crappy CPU)hsd a couple of hours to axfr the zone outI really need to get an SSD RAID into my dev servers, more RAM etc, but it all costs money
Right now I've only got one server that has enough capacity to have ONE vm with enough RAM to have ONE instance of named - and that server has a crappy CPU
Once its signed for the first time, updating it will take a lot less load.
buffrr
commented
1 year ago I got a chance to look at this yesterday. Should be fixed by #7 . Tested with bind. Transfer of 31 million records completed within 21 minutes (although my workstation is a bit overkill 128gb RAM + NVMe storage)
transfer of './IN' from 127.0.0.2#53: Transfer status: success
transfer of './IN' from 127.0.0.2#53: Transfer completed: 65874 messages, 31698494 records, 1078111012 bytes, 1281.099 secs (841551 bytes/sec)so I'm even more thinking its the alg=4
dig doesn't seem to care about alg=4, but since it recognizes supported digest types it complained that mgar DS is claiming SHA256 but the digest exceeds 32 bytes. Oddly, dig ignores digest length < 32 bytes, but bind requires digest length to be correct.
This record was causing bind to fail (but not dig):
bubalusbubalis. 21600 IN DS 10570 8 2 ABE4559EE2A663FA5A6C5B80DB71C69AC4FC4F79BADAD656D0AADFF9 With #7 it ignores those:
[warning] (axfr) DS for bubalusbubalis. has an invalid digest - skippingFiltering the TXT when NS are present is something named will do anyway - so filtering them in this plug-in just saves named from having to handle them, this includes signing them, even tho they never get used!!
If you could make a PR for the TXT/NS stuff that'd be great!
Takes hsd a couple of hours to axfr the zone out
All this encoding in js + iterating over urkel is slow. The plugin could prepare + store a zone file every tree interval in wire format. Then, it can basically just send it over TCP when it gets an AXFR query to make this much faster. With #2 it can also guarantee globally stable SOA serials. PR is up for grabs if you want to do this ;)
Btw are you doing all this just for personal use or are you running a public resolver?
james-stevens
commented
1 year ago dig doesn't seem to care about alg=4
LOL - my dig def does - hex dump is not the "normal" result. I'm running on Alpine, which uses muls-libc, but I wouldn't think that would make any difference - more likely down to a different version of bind, I would think - as the DS validation would be in bind not libc.
hasroot:~# dig +dnssec @192.168.3.165 mgar. any
;; Got bad packet: extra input data
226 bytes
f6 2d 81 00 00 01 00 00 00 03 00 02 04 6d 67 61 .-...........mga
72 00 00 ff 00 01 c0 0c 00 02 00 01 00 00 54 60 r.............T`
00 06 03 6e 73 31 c0 0c c0 0c 00 2b 00 01 00 00 ...ns1.....+....
54 60 00 34 24 30 04 02 8e 4a 39 30 a8 0f 2f da T`.4$0...J90../.
fe da 97 82 f6 9a b2 eb 45 38 d7 f5 ca 23 d8 b4 ........E8...#..
e0 f1 71 73 aa 69 89 9b 7d 0d 09 9b 46 d0 8d 52 ..qs.i..}...F..R
e6 ba 7f 7a 3b 3c 5a d0 c0 0c 00 2e 00 01 00 00 ...z;<Z.........
2a 30 00 53 00 2b 0d 01 00 00 54 60 66 27 b3 54 *0.S.+....T`f'.T
64 45 2e 54 ee 10 00 90 a1 47 6b 98 5e 21 75 de dE.T.....Gk.^!u.
80 6f bb 91 ec bf 28 97 86 7a e3 39 6c f0 52 cf .o....(..z.9l.R.
51 0e fb cb 93 e5 60 60 08 35 ef 7b a1 4e 2b 59 Q.....``.5.{.N+Y
bd 1a 1a 83 c6 49 5c b3 5d 42 50 bd 4a 3b 01 6a .....I\.]BP.J;.j
9a 81 36 0c a5 6b 2b c0 22 00 01 00 01 00 00 54 ..6..k+."......T
60 00 04 cf b4 c6 3a 00 00 29 10 00 00 00 80 00 `.....:..)......
00 00 ..
hasroot:~# named -v
BIND 9.16.33 (Extended Support Version) <id:35e9c6e>But alg=4 is not supported in DNS, so I think filtering that DS is reasonable. Technically, if the DS was allowed through, a DNSSEC enabled resolver (that didn't just completely fail) would not resolve this TLD at all. Presence of the DS means all records MUST be validated, but they are not validatable, so the TLD would fail validation & so not resolve.
for bubalusbubalis my dig reports errors, but still outputs something. Looks like that's exactly what dig has done for the axfr - it has filtered the invalid DS from the text output, which is why my axfr from the text output then works OK.
hasroot:/ext/axfr/ROOT_ZONE# dig +dnssec @192.168.3.165 bubalusbubalis. any
;; Warning: Message parser reports malformed message packet.
; <<>> DiG 9.16.33 <<>> +dnssec @192.168.3.165 bubalusbubalis. any
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57003
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 2
;; WARNING: recursion requested but not available
;; WARNING: Message has 154 extra bytes at end
;; QUESTION SECTION:
;bubalusbubalis. IN ANY
;; AUTHORITY SECTION:
bubalusbubalis. 21600 IN NS ns.bubalusbubalis.my workstation is a bit overkill
yeah, the dev box I'm running this on is garbage, but its the only server I have with enough spare RAM!
Its running off HDD and both named & hsd are performing really badly cos they both do a lot of disk cache-by-passes, on read & write, which takes ages on HDDs - after nearly a week, named has still not been able to sign the entire zone cos it does a disk flush every time it writes to its journal file & its writing one NSEC3 record at a time !!
As the zone is getting so big, its size will probably cause all sorts of problems everywhere. DNS servers mostly run from in-memory databases becuase (in the wild) a large proportion of the queries they get are NXDOMAIN, so will (pretty much) always cause a cache miss - so running from a disk database, with caching, doesn't really help cos a cache miss will usually result in a disk-read anyway - NVMe maybe almost as good as RAM, but SSD probably less so.
When this plug-in walks the tree, there def seems to be some kind of read-without-cache that is causing it to be so slow - nearly all the time is spent in waiting-for-IO.
The reality is, there just aren't that many zones with more than 10M names - so much more than that & you're probably starting to look at needing custom DNS software. I tried knot & nsd (with my signed Handshake ROOT) when the zone was at about 1.5M - they both used about double the RAM of bind, so I went back to bind - I know they both outperform bind for DNSSEC, but I just didn't have the RAM to spare.
All this encoding in js + iterating over urkel is slow.
Have you tried Bun?
Btw are you doing all this just for personal use or are you running a public resolver?
Not sure yet :)
But I did have a working public resolver, until that tool put alg=4 into his zone, so I was kind interested to see why it stopped working.
Unfortunately, resolving Handshake is only half the problem - you also need to solve the HTTPS/CA/DANE issue - And fingertip is kinda lying about validating the DNSSEC data cos I know Namebase have no NS records in most of their zone files, so there's literally no signed NS RRs to validate.
You could def argue that the NS in the blockchain are as good or better, but it's not an RFC validation, that's for sure.
buffrr
commented
1 year ago LOL - my dig def does - hex dump is not the "normal" result.
This isn't failing because alg=4 it's failing because the digest length is 48 bytes but SHA256 is 32 bytes. Try adding alg=4 but correctly sized digest it'll work dig won't care. (I guess more importantly is try #7)
And fingertip is kinda lying about validating the DNSSEC data cos I know Namebase have no NS records in most of their zone files, so there's literally no signed NS RRs to validate.
Fingertip uses libunbound you could file an issue with nlnetlabs if you don't like NB not having authoritative NS in their zones ;) As far as DNSSEC is concerned, a trust path exists.
Have you tried Bun?
It sounds interesting but i doubt it'll make a huge difference.
kind of read-without-cache that is causing it to be so slow - nearly all the time is spent in waiting-for-IO.
I guess we haven't verified if encoding is actually taking the most time. Iterating over urkel may indeed be the cause for this.
As the zone is getting so big, its size will probably cause all sorts of problems everywhere
That's why on the fly signing makes sense for most people. You don't need a super computer to have proper DNSSEC/resolver on your machine/raspberry pi! hsd currently works with all recursive resolvers with on the fly signing. For larger deployments, this plugin is useful. Compact NSECs is turning into an RFC https://www.ietf.org/id/draft-huque-dnsop-compact-lies-01.html
james-stevens
commented
1 year ago This isn't failing because alg=4 it's failing because the digest length is 48 bytes but SHA256 is 32 bytes.
Very odd mine behaves so differently between mgar & bubalusbubalis if they both (essentially) have the same issue??
I've not seen that technique before (NXNAME), its probably the most satisfactory solution to that issue. Instead of creating a new RR type, they could use the unused RR type = 0, often called RR_INVALID
Fingertip uses libunbound you could file an issue with nlnetlabs if you don't like NB not having authoritative NS in their zones ;)
They will fail validation in bind
buffrr
commented
1 year ago Very odd mine behaves so differently between mgar & bubalusbubalis if they both (essentially) have the same issue??
Yeah it's weird. This issue should now be fixed by #7 . Thanks james!
james-stevens
commented
1 year ago I nearly closed #6 the other day :)
Very odd mine behaves so differently between mgar & bubalusbubalis if they both (essentially) have the same issue??
BTW: I meant to say, nice find on bubalusbubalis :+1:
Yeah it's weird. This issue should now be fixed by #7 . Thanks james!
Personally, if this plug-in is going to validate DS records, I think it should retain the filter on alg=4 (i.e. also validate the alg number). Filtering invalid DS records is doing the owner a favour cos it will never validate, so the zone will completely fail to resolve if the end-user asks for validation.
As I recall, alg=4 was dropped in favour of ECDSA cos it has patent issues, so best not get involved in it anyway.
The problem I have is that (AFAIK) the best tool for in-line signed a zone of this size is bind, but the likelihood of bind failing if the data is not RFC compliant is extremely high - as happened before.
This means, if the zone has any invalid data, then I'd have up to the SOA.EXPIRY time to fix the problem (probably by filtering the bad data in this plug-in) before the inline signing bind will drop the ROOT zone - currently that's 6 hours (!!) - the undesirability of this behaviour needs no further explanation.
And the risk of some crappy data messing up bind at some point in the future seems to me basically inevitable.
So I'm also planning to add, to this plug-in, the ability of providing a different SOA in the AXFR explicitly to extend the SOA.EXPIRY. Sadly, the REFRESH & RETRY can be over-ridden in named.conf, but the EXPIRY can't.
Longer term, I'm thinking of using giant-dns to serve out a signed ROOT as its way faster than bind and I think it will have a lower memory requirement - but that needs testing. The benefit there is that I can set the EXPIRY to never, where as bind has a hard-coded limit of 24 weeks ... which is still reasonably long.
BTW: my crappy hardware just finished signing the ROOT zone :rofl:
james-stevens
commented
1 year ago All this encoding in js + iterating over urkel is slow
The most obvious performance improvements I can see are
RRSIGs getting creating - preventing it from signing the data, only to discard the RRSIG, seems an easy win.
After months of faultless operation - the AXFR has broken
the last name that comes out is this