Domain names for Netbird peers not working on Debian 12

[zwn]

Describe the problem I can ping a peer by its IPv4 address but not by its name. When I try, ping says Name or service not known. It works on Windows 11 on a different peer.

To Reproduce Steps to reproduce the behavior:

1. Copy a peer numeric ip from https://app.netbird.io/peers
2. ping \
3. Copy a peer name from https://app.netbird.io/peers
4. ping \

Expected behavior Both work the same.

Additional context Possibly related lines from logfile:

2024-01-19T10:51:05+01:00 ERRO client/internal/dns/server.go:168: got error while restoring the /etc/resolv.conf file from /etc/resolv.conf.original.netbird. Error: got an error while checking stats for /etc/resolv.conf.original.netbird file when copying it. Error: stat /etc/resolv.conf.original.netbird: no such file or directory

2024-01-19T10:51:13+01:00 ERRO client/internal/dns/server.go:282: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured

[zwn]

It seems there is a resolver running from netbird but it does not respond to requests:

$ sudo lsof  -iUDP -P | grep netbird
netbird    807   root   15u  IPv4 212720      0t0  UDP localhost:3128 
netbird    807   root   23u  IPv4 201538      0t0  UDP 100.106.197.71:53 
$ dig @100.106.197.71 <name>.netbird.cloud
;; communications error to 100.106.197.71#53: timed out
...

[pascal-fischer]

Hello @zwn,

we have different implementations for DNS depending on how DNS is managed on your device (e.g. by the network manager, systemd, resolvconf...) From the error message, I can see that it falls back to file-managed DNS. This way we require a nameserver to be configured that resolves all domains for our netbird resolver to work. To fix the issue you need to set a nameserver for all domains in the admin panel. You can assign a distribution group and limit the nameserver to only this peer if required.
Please let me know if this helps.

[zwn]

It does make a difference. When I add the nameserver, the local resolver starts responding to requests. When I tried to query the resolver with dig directly, it used to time out.

However doing plain ping <fqdn> still returns Name or service not known.

This is freshly installed Debian system with no changes. Honestly, I a not even sure which way it is setup. Listing all units by systemctl suggests the system is using neither systemd-resolved nor resolvconf. First line in /etc/resolv.conf says

# Generated by NetworkManager

and the file contains only the resolvers supplied by DHCP.

Trying out nmcli c show wt0 | grep dns show there is no resolver and no search domain associated with the connection.

What else can I try?

[mlsmaycon]

Hello @zwn it seems like there are two issues. First, the agent is not adding the configuration to the NetworkManager, and second, there seems to be an issue with connectivity with the DNS listener.

Can you please enable debug logs with:

sudo netbird service stop
sudo netbird service uninstall
sudo netbird service install --log-level debug
sudo netbird service start

Please share the new logs with us. (you can mask the IPs if you like it, too.

Can you also share the output of nftables: nft list ruleset?

[zwn]

Interesting. I have run the steps to enable debug logs and now.... it works. The contents of /etc/resolv.conf are now:

# Generated by NetBird
# If needed you can restore the original file by copying back /etc/resolv.conf.original.netbird

search netbird.cloud
nameserver 100.106.197.71
nameserver 1.1.1.1
nameserver 1.0.0.1

Oh, and I don't have nftables

$ nft
bash: nft: command not found

So I am all set now. If there is anything else I can do to help find the root cause of the problem, just let me know. I have a freshly installed Debian 12 system and to get netbird I have followed the steps at https://app.netbird.io/install which resulted in me being able to use only numeric IPs to get to the peers. Then I have added cloudflare nameserver in the admin console and removed and re-added the service as advised, after which everything works as expected.

[zwn]

It seems to be related to sleep (suspend). After wakeup, /etc/resolv.conf is back to NetworkManager generated but netbird status says I am connected. Bring it down and up again gives me back Generated by NetBird resolv.conf - so far so good - but ping <name> still does not work. The netbird resolver works (responds to dig @<nebirdip> <name> correctly) but the system does not use it despite being listed in the resolv.conf now. After another down & up it works again. So maybe time for some logs? Which ones and how much?

[jorgeag68]

Same issue is happening on ubuntu 22.04.
Cannot get the DNS to work.

2024-04-04T09:22:03+01:00 ERRO client/internal/dns/server.go:323: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured 2024-04-04T09:22:03+01:00 INFO client/internal/acl/manager.go:52: ACL rules processed in: 1.045913ms, total rules

[lixmal]

@zwn can you check if the issue still persists with newer versions? We've implemented a constant-overwrite for the file manager.

[lixmal]

@jorgeag68 have you followed this comment https://github.com/netbirdio/netbird/issues/1483#issuecomment-1904282825

[jorgeag68]

@lixmal Thank you for pointing that out to me. I've configured a generic DNS for group All and then the specific DNS depending on other groups/locations. It now works as expected on ubuntu 22.04.

[zwn]

@lixmal Seems to be working now. Setting up the global nameserver as described https://github.com/netbirdio/netbird/issues/1483#issuecomment-1904282825 is required - but it is somewhat surprising - I am not sure how I would get this info were it not for this comment.

[cannonfodda]

@zwn I'm with you on that one. I've spent days trying to work around this and if I hadn't stumbled across this issue I'd still be going. @lixmal It would be great to add this to the docs for those working on minimum installs without resolved.

[lixmal]

We mention it here https://docs.netbird.io/how-to/manage-dns-in-your-network#match-domains in the hint box but I guess it could be improved

[cannonfodda]

@lixmal Gotcha thanks👍. I had seen that hint but given I wasn't trying to do anything fancy with nameservers or the like it never occurred to me that it might be the solution. We are working off a minimal debian install so it's only NetworkManager that is set up by default and I had assumed that local resolution would work out of the box as that's implied by the docs. It would good to add a statement to the docs for us simple people who aren't planning on running their own nameservers. 😄

Happy to help draft something if it's of use.