Problems with tailscale and MagicDNS

[ElnuDev]

Describe the bug

I have a self-hosted instance of headscale and with client nodes, all on NixOS. They work fine, but for some reason I can't get MagicDNS to work at all. I found this discussion on the tailscale forum that mentioned adding the tailscale route to networking.search, but that still does nothing.

I've tried:

networking.search = [ "headscale.elnu.com" ];

I'm not sure if this is a bug or not, but a part of the issue may potentially be that my server URL is headscale.elnu.com and my DNS base domain is simply elnu.com?

Either way, when I run tailscale status, it outputs the following at the bottom:

# Health check:
#     - dns-os: exit status 1
#     - dns: exit status 1

My configuration for headscale is here and my configuration for the tailcale client is here.

Sorry for making an issue about this, but I'm honestly just not sure where else to ask because of this issue being an intersection between tailscale/headscale and NixOS.

Thanks in advance!

Notify maintainers

@danderson @martinbaillie @twitchyliquid64 @06kellyjac

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[elnu@elnu:~/nix-config]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.4.6, NixOS, 23.11 (Tapir), 23.11.20230801.9e1960b`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.15.1`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[twitchyliquid64]

What DNS server are you using in your clients' nix config?

systemd-resolvd is probably what you need if you havent got it already.

[ElnuDev]

Oh hold on. I was looking back through my config and realized a while back I set networking.resolvconf.enable = false; because it wasn't messing with setting my nameservers. Let me try enabling it and see if MagicDNS starts working.

[ElnuDev]

Alright, enabling networking.resolvconf fixed the issue, but for some reason MagicDNS is only working for fully-qualified tailnet domain names, not hostnames. Any ideas why? Thanks for the help!

[twitchyliquid64]

but for some reason MagicDNS is only working for fully-qualified tailnet domain names, not hostnames

Did you configure the search domains in Tailscales admin panel? (no idea how to do that on headscale)

Also confirming you are testing using nslookup or the browser instead of something like dig (which doesnt support the shortcut hostnames using search domains thing) ?

[ElnuDev]

headscale search domains are defined by services.headscale.settings.dns_config.domains and I don't have that set, let me give that a try.

Regarding testing, I've just been testing by trying to access the hostnames in Firefox and trying to SSH in.

[ElnuDev]

Well, I've set services.headscale.settings.dns_config.domains = [ "elnu.com" ] on the headscale server (which is the base domain, headscale is running at headscale.elnu.com). I've tried setting networking.search = [ "elnu.com" ] as well and it's not making any difference. Any ideas?

[ElnuDev]

Alright, I resolved the issue! There was in fact nothing wrong, I just didn't realize that all the devices had to be in different namespaces in order to be able to talk to each other via direct hostnames. Now that I have them all under the same namespace/user, it all works fine. Thanks for the help, sorry for the trouble!

[ElnuDev]

@twitchyliquid64 Alright, it's not exactly the same issue as before, but since it's related, I'm going to reopen this issue. (Sorry for the trouble.)

Alright, enabling networking.resolvconf fixed the issue

I forgot the reason why I originally disabled resolvconf was because when it was enabled, it would completely ignore my nameserver configuration in networking.nameservers. After some investigation, I found that in resolvconf.conf there's a configuration parameter name_servers which isn't modified by networking.nameservers. After trying the following configuration

networking = {
  networkmanager.enable = true;
  resolvconf.extraConfig = "name_servers=\"9.9.9.9 1.1.1.1\"";
};

I managed to get my desired DNS working without any problem! However, somehow this configuration of explicitly defining name_servers breaks MagicDNS, and even stranger, the health check dns and dns-os fails are not present in tailscale status. I have all of my devices under the same user and have checked both the hostname and fully qualified domain for devices and both cannot resolve. I've of course tried rebooting multiple times and the issue still persists.

Any ideas what's going on? Not sure if I'm doing something dumb here, but perhaps I should open a new issue. The entire resolvconf (which is enabled by default) not playing nice with explicitly defined nameservers unless using extraConfig rather than networking.nameservers seems odd as well, and might be an oversight.

[06kellyjac]

Tailscale is changing your resolv.conf

https://tailscale.com/kb/1235/resolv-conf/

This isnt a nixos issue

Have you tried setting 9.9.9.9 as your global dns in tailscale and enabling tailscale override local dns?

E.g. like so but for q9

https://tailscale.com/kb/1218/nextdns/

[lorenzleutgeb]

For anyone stumbling upon this issue, I strongly recommend reading "The Sisyphean Task Of DNS Client Config on Linux" (Tailscale blog) before configuring anything.

I have since switched to systemd-resolved, but in the past I configured NetworkManager's dnsmasq to use Tailscale:

{ config, lib, ... }:
let
  tailnetName = builtins.throw "TODO: Add the name of your tailnet, here. Should look like 'example.ts.net'";
  tailscaleDns = "100.100.100.100";
in {
  environment.etc."NetworkManager/dnsmasq.d/${tailnetName}.conf" = lib.mkIf (config.networking.networkmanager.dns == "dnsmasq") {
    text = ''
      server=/${tailnetName}/${tailscaleDns}
      server=${tailscaleDns}@${config.services.tailscale.interfaceName}
    '';
  };
}