DNS issues for custom DNS servers/IPs (write: can't assign requested address)

[pnowy]

Describe the problem

We are using NetBird for some time for group of people but sometimes some users reports the problem with DNS (we have 2 internal zones within GCP/AWS clouds, added conditionally in NetBird).

To Reproduce

Problem exist only for selected users. Cannot easily be reproduced. What's DNS logs showing for the problematic user:

OS: darwin/arm64
Daemon version: 0.32.0
CLI version: 0.32.0
Management: Connected to https://intranet.example.com:33073
Signal: Connected to http://intranet.example.com:10000
Relays: 
  [stun:intranet.example.com:3478] is Available
  [turn:intranet.example.com:3478?transport=udp] is Available
Nameservers: 
  [172.31.0.2:53] for [internal-example-cloud1.com] is Unavailable, reason: 1 error occurred:
    * write udp 192.168.0.102:50905->172.31.0.2:53: write: can't assign requested address
  [1.1.1.1:53, 1.0.0.1:53] for [.] is Available
  [8.8.8.8:53, 8.8.4.4:53] for [.] is Available
  [172.25.16.7:53, 10.100.0.18:53] for [internal-example-cloud2.com] is Unavailable, reason: 2 errors occurred:
    * write udp 10.90.211.76:61742->10.100.0.18:53: write: can't assign requested address
    * write udp 192.168.0.102:59568->172.25.16.7:53: write: can't assign requested address
FQDN: laptop-51896.netbird.selfhosted
NetBird IP: 100.77.202.187/16
Interface type: Userspace
Quantum resistance: false
Routes: -
Peers count: 3/3 Connected

Expected behavior

The connectivity to our routers works fine. The user reach the server and execute request (e.g. by adding DNS Host header) but the problem is DNS. It looks like for some reason selected clients have write: can't assign requested address when trying to write that UDP.

Are you using NetBird Cloud?

Self-host NetBird's control plane (0.28.9).

NetBird version

Client version: 0.32.0 (but user had this in 0.28.9 - the same as server, just upgraded to latest to check if problem exist).

NetBird status -dA output:

For non-problematic user DNS status looks like:

OS: darwin/arm64
Daemon version: 0.30.0
CLI version: 0.30.0
Management: Connected to https://intranet.anon-FDRBR.domain:33073
Signal: Connected to http://intranet.anon-FDRBR.domain:10000
Relays:
  [stun:intranet.anon-FDRBR.domain:3478] is Available
  [turn:intranet.anon-FDRBR.domain:3478?transport=udp] is Available
Nameservers:
  [172.31.0.2:53] for [anon-lOBIO.domain] is Available
  [1.1.1.1:53, 1.0.0.1:53] for [.] is Available
  [8.8.8.8:53, 8.8.4.4:53] for [.] is Available
  [172.25.16.7:53, 10.100.0.18:53] for [internal.anon-FDRBR.domain, gcp.anon-hIzBU.domain] is Available
FQDN: laptop-xyz.netbird.selfhosted
NetBird IP: 100.77.68.36/16
Interface type: Userspace
Quantum resistance: false
Routes: -
Peers count: 3/3 Connected

Any recommendation what could be the issue are welcome (I suspect this is some with different DNS handling for different client).

[mgarces]

Hi, can you share the output for netbird status -dA --filter-by-ips 10.100.0.18 from the peer that is having DNS (assuming 10.100.0.18 is the DNS peer).

[pnowy]

Requested the problematic client for details, will put here when get them.