NixOS / infra

NixOS configurations for nixos.org and its servers
MIT License
214 stars 94 forks source link

IPv6-only recursive resolving of cache.nixos.org broken #158

Open mweinelt opened 2 years ago

mweinelt commented 2 years ago

Recursively resolving cache.nixos.org in an IPv6-only setup is impossible, since the fastly.com authoritative nameservers don't provide IPv6 connectivity.

# drill -T cache.nixos.org AAAA
.   518400  IN  NS  a.root-servers.net.
.   518400  IN  NS  b.root-servers.net.
.   518400  IN  NS  c.root-servers.net.
.   518400  IN  NS  d.root-servers.net.
.   518400  IN  NS  e.root-servers.net.
.   518400  IN  NS  f.root-servers.net.
.   518400  IN  NS  g.root-servers.net.
.   518400  IN  NS  h.root-servers.net.
.   518400  IN  NS  i.root-servers.net.
.   518400  IN  NS  j.root-servers.net.
.   518400  IN  NS  k.root-servers.net.
.   518400  IN  NS  l.root-servers.net.
.   518400  IN  NS  m.root-servers.net.
org.    172800  IN  NS  a0.org.afilias-nst.info.
org.    172800  IN  NS  a2.org.afilias-nst.info.
org.    172800  IN  NS  b0.org.afilias-nst.org.
org.    172800  IN  NS  b2.org.afilias-nst.org.
org.    172800  IN  NS  c0.org.afilias-nst.info.
org.    172800  IN  NS  d0.org.afilias-nst.org.
nixos.org.  86400   IN  NS  dns1.p02.nsone.net.
nixos.org.  86400   IN  NS  dns2.p02.nsone.net.
nixos.org.  86400   IN  NS  dns3.p02.nsone.net.
nixos.org.  86400   IN  NS  dns4.p02.nsone.net.
cache.nixos.org.    3600    IN  CNAME   dualstack.v2.shared.global.fastly.net.
net.    172800  IN  NS  b.gtld-servers.net.
net.    172800  IN  NS  k.gtld-servers.net.
net.    172800  IN  NS  c.gtld-servers.net.
net.    172800  IN  NS  i.gtld-servers.net.
net.    172800  IN  NS  a.gtld-servers.net.
net.    172800  IN  NS  g.gtld-servers.net.
net.    172800  IN  NS  l.gtld-servers.net.
net.    172800  IN  NS  e.gtld-servers.net.
net.    172800  IN  NS  d.gtld-servers.net.
net.    172800  IN  NS  h.gtld-servers.net.
net.    172800  IN  NS  j.gtld-servers.net.
net.    172800  IN  NS  m.gtld-servers.net.
net.    172800  IN  NS  f.gtld-servers.net.
fastly.net. 172800  IN  NS  ns1.fastly.net.
fastly.net. 172800  IN  NS  ns2.fastly.net.
fastly.net. 172800  IN  NS  ns3.fastly.net.
fastly.net. 172800  IN  NS  ns4.fastly.net.

Debian, they also host their cache at fastly, have their CNAME set to something below fastlydns.net, which does have full IPv6 connectivity.

# drill -T deb.debian.org AAAA
.   518400  IN  NS  a.root-servers.net.
.   518400  IN  NS  b.root-servers.net.
.   518400  IN  NS  c.root-servers.net.
.   518400  IN  NS  d.root-servers.net.
.   518400  IN  NS  e.root-servers.net.
.   518400  IN  NS  f.root-servers.net.
.   518400  IN  NS  g.root-servers.net.
.   518400  IN  NS  h.root-servers.net.
.   518400  IN  NS  i.root-servers.net.
.   518400  IN  NS  j.root-servers.net.
.   518400  IN  NS  k.root-servers.net.
.   518400  IN  NS  l.root-servers.net.
.   518400  IN  NS  m.root-servers.net.
org.    172800  IN  NS  d0.org.afilias-nst.org.
org.    172800  IN  NS  a0.org.afilias-nst.info.
org.    172800  IN  NS  c0.org.afilias-nst.info.
org.    172800  IN  NS  a2.org.afilias-nst.info.
org.    172800  IN  NS  b0.org.afilias-nst.org.
org.    172800  IN  NS  b2.org.afilias-nst.org.
debian.org. 86400   IN  NS  nsp.dnsnode.net.
debian.org. 86400   IN  NS  dns4.easydns.info.
debian.org. 86400   IN  NS  sec1.rcode0.net.
debian.org. 86400   IN  NS  sec2.rcode0.net.
deb.debian.org. 3600    IN  CNAME   debian.map.fastlydns.net.
net.    172800  IN  NS  a.gtld-servers.net.
net.    172800  IN  NS  b.gtld-servers.net.
net.    172800  IN  NS  c.gtld-servers.net.
net.    172800  IN  NS  d.gtld-servers.net.
net.    172800  IN  NS  e.gtld-servers.net.
net.    172800  IN  NS  f.gtld-servers.net.
net.    172800  IN  NS  g.gtld-servers.net.
net.    172800  IN  NS  h.gtld-servers.net.
net.    172800  IN  NS  i.gtld-servers.net.
net.    172800  IN  NS  j.gtld-servers.net.
net.    172800  IN  NS  k.gtld-servers.net.
net.    172800  IN  NS  l.gtld-servers.net.
net.    172800  IN  NS  m.gtld-servers.net.
fastlydns.net.  172800  IN  NS  ns1.fastlydns.net.
fastlydns.net.  172800  IN  NS  ns2.fastlydns.net.
fastlydns.net.  172800  IN  NS  ns3.fastlydns.net.
fastlydns.net.  172800  IN  NS  ns4.fastlydns.net.
debian.map.fastlydns.net.   30  IN  AAAA    2a04:4e42:62::644
fastlydns.net.  86400   IN  NS  ns1.fastlydns.net.
fastlydns.net.  86400   IN  NS  ns2.fastlydns.net.
fastlydns.net.  86400   IN  NS  ns3.fastlydns.net.
fastlydns.net.  86400   IN  NS  ns4.fastlydns.net.

Can we find out what this is, and how we can get it, too?

zimbatm commented 2 years ago

I'm not sure if that's possible. debian.map.fastlydns.net. looks like something custom they got from Fastly.

https://support.fastly.com/hc/en-us/articles/360035069912-IPv6-support doesn't mention fastlydns.

zimbatm commented 2 years ago

I agree that it's weird that ns1.fastly.net doesn't reply AAAA. Luckily, most of the time I would expect an intermediate DNS to reply to the query but still.

mweinelt commented 2 years ago

The problem is that ns1.fastly.net is not reachable via IPv6, not that it does not reply with a AAAA record (which it does for me).

❯ echo ns{1,2,3,4}.fastly.net | xargs -n 1 host -t AAAA
ns1.fastly.net has no AAAA record
ns2.fastly.net has no AAAA record
ns3.fastly.net has no AAAA record
ns4.fastly.net has no AAAA record
vcunat commented 2 years ago

I don't know... so poke their support? https://support.fastly.com

It might work even without any customer account. Any better ideas? EDIT: I did look into their docs further and found nothing.

vcunat commented 2 years ago

Query sent.

vcunat commented 2 years ago

I'll need our account ID, apparently.

We may be able to squeeze you into our IPv6 authoritative DNS delivery beta program, If you're happy to do so. You will need to agree to our Terms of Service conditions which should be with you soon.

Can you confirm your account ID please.

EDIT: I hope "beta" doesn't mean anything risky really.

EDIT2: we're following up the support thread now.

vcunat commented 2 years ago

Nothing risky is involved but we would like to make you aware of the fact that performance is not as tuned as for our IPv4 only DNS answers so you may see something there. You will also need to work with us to provide us with insight into any performance issues you may see. We will use that insight to help improve the performance. If you're are ok with this we are happy to include you.

Sounds OK to me, but I expect that also some else should ACK it before proceeding.

I don't expect that speed of DNS itself could be as significant for us in this case, as there are few names and will be mostly used in large batches (amortization through caching).

vcunat commented 2 years ago

Hmm, their "beta" wording isn't as encouraging as I hoped, e.g.

Fastly strongly advises against using production traffic for Beta products due to their dynamic nature.

zimbatm commented 2 years ago

Yeah, let's wait.

grahamc commented 1 year ago

Happy 1 year anniversary of this issue. Just contacted Fastly support about this to see what they say.

Update: they said I'd hear back tomorrow.

grahamc commented 1 year ago

IPv6 has been enabled on all our distributions. However, it involves a configuration change in our DNS. I'm confirming with Fastly that we should in fact replace the CNAMEs with A's and AAAA's on our end.

Kabbone commented 1 year ago

are there any news on this topic?