Moving to another DNS provider

[wdhdev]

(reference #13995)

---

Alternatives

Cloudflare

• Pros
  • Free, up until certain limits
  • Cloudflare proxy option
  • Caching via Cloudflare's global CDN (when proxy is enabled)
  • Faster DNS deployment
  • Built-in DDoS protection for subdomains (when proxy is enabled)
  • NS record support for subdomains
  • Free email forwarding (can setup support@is-a.dev, security@is-a.dev, etc)
  • Cloudflare Zero Trust
    • Tunnels
• Cons
  • Record limit, although can be increased by contacting support I believe
  • Not great support

Bunny.net

• Pros
  • Good support
  • Custom/whitelabel nameserver support (e.g. ns1.is-a.dev & ns2.is-a.dev show as the nameserver records, but Bunny.net is the backend)
  • DNS query logs and statistics
  • Importing zone file
• Cons
  • Paid
  • Limit of 20 million queries a month, before it's paid
  • No NS record support (also has limited records types available, compared to Cloudflare)

---

Overall cons

• Moving CI over could be a hassle, however using DNSControl could make it easier. (used with Open Domains and Free Domains)
• URL records, not sure how we would manage to implement these, but we could figure out a way.

[MaskDuck]

I can write an URL redirector in Python (however would not be a preferable solution, the more preferable one would be you writing one on JS or see the next line) however I think you can also deprecare URL records (maybe send an email to people with URL records to tell them to remove before X?)

CF rate limit: "Record limit, although can be increased by contacting support I believe"

the number of records we have now has already surpassed the limit of 1000 for free and 3500 for paid.

[wdhdev]

I can write an URL redirector in Python (however would not be a preferable solution, the more preferable one would be you writing one on JS or see the next line) however I think you can also deprecare URL records (maybe send an email to people with URL records to tell them to remove before X?)

We can do this, and just setup a VPS for it to run on.

the number of records we have now has already surpassed the limit of 1000 for free and 3500 for paid.

Cloudflare support can help with this. JS.ORG exceeds that limit I'm pretty sure, and they are on the free plan iirc.

[MaskDuck]

but we have to reach out for support before the migration

[wdhdev]

Yep, that's true.

[MaskDuck]

[zvdxcsite]

I found something called deSec, which you can also self-host, and haven´t found a limit. https://github.com/desec-io/desec-stack https://desec.io/

[wdhdev]

@zvdxcsite If we did self host our DNS, it would most likely be with a more advanced and more well-known project.

[NullyIsHere]

Also it should be a really good one and good server cause it would probably crash.

[github-actions[bot]]

This issue has been marked as stale due to inactivity and will be closed. Comment anything on this issue to prevent it

[wdhdev]

@phenax After some research (and testing), I think if we did self host our DNS it would be quite beneficial, the only thing is we would need to get at least 2 VPS' with 100% uptime (in 2 different locations, maybe one in the US and another in EU?). Another thing we would need is API support.

After testing Technitium DNS Server, I've found it would probably would work very well. I have personally tried running this and the DNS propagation is extremely fast, and the software itself supports many different record types (including NS) and is super easy to use and understand. With little to no experience in hosting DNS servers, I managed to quite easily host Technitium and setup a DNS zone and got it all up in running with-in <30 minutes. It also has a HTTP API, which would be perfect for our project.

The only main downsides to this is basically learning the API, although I wouldn't think it would be too hard and possibly DNS resolution times would be a bit slower than using a provider like Namecheap or Cloudflare. The software requirements are extremely low, so cost would be very low to run (possibly cheaper than CPanel?). As we use Namecheap as our domain registrar, registering nameservers is very easy.

The reason for 2 VPS' would be one root nameserver and then a secondary DNS server so in the event the root nameserver goes down, the secondary nameserver can still resolve the DNS zone.

[NullyIsHere]

Seems cool, @wdhdev since I read that i started to work on something for URL records, its almost finish.

[wdhdev]

@NullyIsHere Sounds good, let me know when you're finished!

[NullyIsHere]

Status update: Working on a script to detect URL records, where and in what domain.

[phenax]

A few points to consider here:

• URL redirection is a separate problem to DNS so it can be dealt with separately. So instead of trying to migrate everything at once, we can simplify things a lot by solving 1 problem at a time.
• It should have preferably no (or at least very high) limits on the number of dns records (and redirections) or rate-limits on the api to manipulate those records.
• Github should not be the source of truth for data for either of them. I.e. no fetching records directly from github. The records should be saved somewhere else in a more efficient/indexed form.
• Prefer fixed pricing for our solutions. Since it's more predictable.
• It should support at least all of our current features.
• Scaling is a pretty messy problem and I have no idea when scaling becomes a concern for dns servers. If we're going with a self-hosted solution, we'll need to perform extensive load testing and we'll need really good monitoring tools.
• If we're relying on a managed service, customer service should be really good or else we're stuck dealing with problems we have no control over.
• Security will be hard to validate but we'll need to keep an eye on it and be as conservative about access as possible.

@wdhdev, Technitium looks great. Appreciate the thorough research! I'll experiment with it a bit this week. If that doesn't work out bunny.net seems like a decent managed alternative (zone management, not the scriptable dns). The pricing is not fixed unfortunately but it seems reasonable enough.

No NS record support (also has limited records types available, compared to Cloudflare)

It seems like bunny supports it here but I'm not sure if there are any implicit restrictions on it.

[wdhdev]

@phenax I've done some testing with Technitium and so far, it seems very good. It even allows you to host your own local domains and TLDs if wanted. It has easy to setup secondary nameservers as well (only a few clicks to setup). It also has in-depth statistics.

Thanks for your response!

[MaskDuck]

also uhm... 20 million queries a month for bunny... in my opinion we will need to like uhm get some stats about the queries: currently are we exceeding 20m queries a month? I doubt that but we need to make sure about that if we're moving to bunny.

[wdhdev]

@MaskDuck @phenax For hosting, uptime and reliability this is what I personally think we would/should do:

• Main server (ns1.is-a.dev)
  • Hosted on a VPS provider like OVHcloud in the United States
• Secondary server (ns2.is-a.dev)
  • Hosted on the same provider, but somewhere else in the EU, like Germany or the UK.
• Possible 3rd server (ns3.is-a.dev) (optional)
  • Also hosted on the same provider, but in Australia.
• And so on...

This way, if the main nameserver, ns1.is-a.dev goes down, ns2.is-a.dev and ns3.is-a.dev (if it exists), will just pickup the load immediately causing no downtime for DNS queries.

TL;DR; Have 2 or 3 servers in different regions to minimise loading times for users in different regions and to avoid downtime if a server goes down, therefore increasing reliability.

The reason why I recommended OVHcloud is because they have a high SLA for most services and notice in advance for maintenance.

[MaskDuck]

I'd recommend you two host two different nameservers on two different platforms

[wdhdev]

I'd recommend you two host two different nameservers on two different platforms

Same provider, different datacenters will be good enough for our needs, in my opinion. Also, 2 nameservers should be enough, the main one in the US, a secondary one in the EU.

[NullyIsHere]

Germany could be a good option as it could cover parts of the East

[wdhdev]

Germany could be a good option as it could cover parts of the East

Yeah, I was thinking Germany.

[andrewstech]

Germany could be a good option as it could cover parts of the East

Yeah, I was thinking Germany.

The server that used to run hosting and manage site is Located in London, England that could also become part of this as it only running the discord bot and open-domains plus it has a 25GB uplink

[wdhdev]

The server that used to run hosting and manage site is Located in London, England that could also become part of this as it only running the discord bot and open-domains plus it has a 25GB uplink

We will need DDoS protection & stuff. Unfortunately, we probably won't be able to use your server unless Phenax wants to of course.

[phenax]

I might be wrong but from what I can tell, Technitium only works on disk. Which means managing multiple instances with that will be a shit show. Both on how we update dns records across instances and how we scale instances.

We'll need a common data source that lives outside disk. This is because killing an instance should not mean all the records are gone and we now have the headache of migrating the records on disk to the new instance before that. Which we may not be able to do reliably. So we'll need 2 instances running the dns server and 1 db instance with backups.

If the caching strategy isn't good enough, a ddos attack could potentially overload the db. But the chance of that is pretty slim and can be dealt with later with replication. We'll have a better idea after some stress testing and tweaking our dns solution a bit.

PowerDNS seems like a pretty strong alternative that supports external backends for storage and has decent caching which means we can deploy multiple instances of this and scale it without much hassle. Also has an api and a dashboard. Will try this out later.

Let me know what you all think.

[wdhdev]

@phenax For Technitium, you just setup 2 or more instances, run the zone on, ns1.is-a.dev, then configure secondary zones on the other nameservers, which point to ns1.is-a.dev, after adding NS records for each secondary nameserver in the primary zone. The zone is then automatically synced to secondary servers.

I have my own setup using Technitium including a secondary nameserver and I'm happy to show you how I have it configured. Me and @MaskDuck were just doing some testing with it a little while ago. I can hop in a Discord call with you if you want, anytime today, just DM me or ping me in the support server.

I have seen PowerDNS before, however it seems a bit complicated to setup, at least for me.

Also, in my opinion, it is better to use a disk backend. The reason for this is because that way if the main Technitium instance goes down (or if using PowerDNS, the external backend), it will not disrupt traffic at all, as the zone is striped across all secondary nameservers.

[wdhdev]

@phenax I've sent you a message on Discord showing you how to manage Technitium with multiple instances and some recommended options to enable, with screenshots. It is super easy to setup and to scale. The main downside is just having a login for each instance, other than that, it is super easy to scale.

[andrewstech]

The server that used to run hosting and manage site is Located in London, England that could also become part of this as it only running the discord bot and open-domains plus it has a 25GB uplink

We will need DDoS protection & stuff. Unfortunately, we probably won't be able to use your server unless Phenax wants to of course.

My server is now equipped with DDOS protection, Upgraded my plan last month. Also all servers must have an IPV6 address otherwise AAAA records won’t be resolved

[wdhdev]

My server is now equipped with DDOS protection, Upgraded my plan last month.

Alright then, we can most likely configure it as a secondary nameserver, the more nameservers the better 😆

I might consider hosting one as well.

Also all servers must have an IPV6 address otherwise AAAA records won’t be resolved

I've just tested my Technitium instance, which does not have an IPv6 assigned and AAAA records do in fact resolve. You can see the output here:

$ dig ipv6.test.wdh.gg AAAA +noall +answer
ipv6.test.wdh.gg.       3600    IN      AAAA    2001:4860:4860::8888

[NullyIsHere]

One problem i thought about is that ns cant be n1.is-a.dev as to resolve that needs to resolve n1.is-a.dev, etc. So it should be n1.wdh.gg oor similar

[andrewstech]

One problem i thought about is that ns cant be n1.is-a.dev as to resolve that needs to resolve n1.is-a.dev, etc. So it should be n1.wdh.gg oor similar

We can use glue records to resolve this. They are configured on the domain before the nameserver. They sort of sit on top

https://www.ibm.com/blog/understanding-glue-records-and-dedicated-dns/

[wdhdev]

One problem i thought about is that ns cant be n1.is-a.dev as to resolve that needs to resolve n1.is-a.dev, etc. So it should be n1.wdh.gg oor similar

Glue records is the answer as @andrewstech said before. These are configured at the registrar level, most (if not all) major registrars like Namecheap support them.

[NullyIsHere]

Thanks for the explanation. :)

[wdhdev]

@NullyIsHere Hey, are you able to join our Discord server to discuss the URL redirection script?

[NullyIsHere]

Yeah, ill join now.

[phenax]

The zone is then automatically synced to secondary servers.

That sounds alright. I just have a painful past of debugging systems with multiple sources of truth. Shouldn't be a huge deal either way.

I have seen PowerDNS before, however it seems a bit complicated to setup, at least for me.

This shouldn't be a factor since we won't be managing our infrastructure by hand every time.

that way if the main Technitium instance goes down (or if using PowerDNS, the external backend), it will not disrupt traffic at all

Not an issue with an external backend because they can be scaled independently of the instances with a read-only replica.

@wdhdev The problem I see with Technicium is that it doesn't seem like a production-grade DNS solution. They seem to advertise themselves as a home/org network solution and ad blocker like pi-hole which doesn't give me any confidence. Feels like forcing a fit. Let me know what you think.

But yeah, I guess either on-disk or external should work fine for us. Might be worth looking into more solutions.

[wdhdev]

Thanks for your response, I do agree it is more personal use and meant for more smaller networks, so maybe it isn't the solution for us.

PowerDNS does seem like it is advertised more for bigger applications like ours. I might spin up a PowerDNS instance later today and send a few screenshots either here or on Discord.

[andrewstech]

That sounds alright. I just have a painful past of debugging systems with multiple sources of truth. Shouldn't be a huge deal either way.

If we did take the PowerDNS route then we can use the CI from Open Domains. As that uses DNSControl to automate record deployments as it has support for PowerDNS.

CI: https://github.com/open-domains/register/blob/main/.github/workflows/publish.yml

Config: https://github.com/open-domains/register/blob/main/dnsconfig.js

Would be very easy to adjust to is-a.dev file structure

[wdhdev]

PowerDNS does seem like it is advertised more for bigger applications like ours. I might spin up a PowerDNS instance later today and send a few screenshots either here or on Discord.

I got a bit lazy and forgot about this. I tried installing a PowerDNS instance with PowerDNS-Admin however for whatever reason I could not get it working for whatever reason (@Stef-00012 and @CuteDog5695 can confirm this as they were watching my screenshare on Discord). @phenax if you could try spin up an instance that might be better.

[phenax]

@wdhdev I have a test setup for powerdns with pdns-admin. Pushed it in an infra repo. We'll use that as a base for our infra.

[wdhdev]

@phenax Oh, amazing. I couldn't get PowerDNS-Admin working myself so that's good. Are you able to give me a login to the server possibly? I'd like to run some of my own tests on one of my test domains.