search

TechnitiumSoftware / DnsServer

Technitium DNS Server
https://technitium.com/dns/
GNU General Public License v3.0
3.86k stars 401 forks source link

[Feature Request] - Multiple IPs for Resolution #760

Closed uppaljs closed 4 months ago

uppaljs commented 8 months ago

I am looking to use technetium DNS Server in a resolver cluster, I would like to know how do we use multiple ips when doing recursion. e.g. my interface has 15 IPs assigned, how do I configure the DNS Server to use these 15 Ips for sending queries fore recursion. The inbound requests come over private network which is on another interface.

Use Case:

Currently Technitium only picks up the default route and uses that to send recursive queries to the internet. If you have a lot of load , some of the upstream servers start throttling your Recursor IP so that use case is to have multiple IPs on the internet and the DNS server send queries outbound using them by a configuration parameter.

Just as query-local-address in PowerDNS recursor

https://doc.powerdns.com/recursor/settings.html#query-local-address "Send out local queries from this address, or addresses. By adding multiple addresses, increased spoofing resilience is achieved. When no address of a certain address family is configured, there are no queries sent with that address family. In the default configuration this means that IPv6 is not used for outgoing queries."

ShreyasZare commented 8 months ago

Thanks for the feature request. Will get this option added soon.

sellers commented 5 months ago

@uppaljs - are you able to get your resolver to respond on the virtual/additional IP addresses on your host? I can't seem to get it to respond to anything other the the primary IP and the loopback. (e.g. on Linux I can't seem to get it to respond to requests to the secondary IP on the same interface but the website responds and TCP connection responds and LSOF shows the UDP is listening on * )

Edit: I just realized what is meant by the multiple IP address note for Linux, i was using 0.0.0.0:53 and that does not seem to play nice so I explicitly listed them as suggested and it appears to be more happy. Dropping this here in case someone else runs across this.

ShreyasZare commented 4 months ago

Technitium DNS Server v12 is now available that adds option to specify one or more source addresses or network for making outbound DNS requests. Do update and let me know your feedback.

tannyl commented 4 months ago

I have tried using this new feature but it does not seem to work as expected. I am running the server in a Docker container.

Here is an example Docker Compose configuration to help illustrate the problem. We will pretend that the docker networks are connected to a firewall which block all DNS queries on vlan 10 but allow them on vlan 20.

version: "3"
services:
  dns-server:
    container_name: dns-server
    image: technitium/dns-server:latest
    networks:
      recursiveBlocked:
        ipv4_address: 10.10.10.2
      recursiveAllowed:
        ipv4_address: 10.10.20.2

networks:
  recursiveBlocked:
    name: a_eth0_10
    driver: macvlan
    driver_opts:
      parent: eth0.10
    ipam:
      config:
        - subnet: 10.10.10.0/24
          gateway: 10.10.10.1

  recursiveAllowed:
    name: b_eth0_20
    driver: macvlan
    driver_opts:
      parent: eth0.20
    ipam:
      config:
        - subnet: 10.10.20.0/24
          gateway: 10.10.20.1

In this configuration the container will end up with two interfaces inside it:

The gateway 10.10.10.1 on eth0 is the default inside the container. The reason for this is due to the naming of the two Docker networks. The order of interfaces inside containers is based on the alphabetical order of the attached non-internal networks interfaces names.

When this container is started the dns server won't be able to do recursive queries due to it using the default gateway on eth0, which is connected to vlan 10 where DNS queries are blocked.

If you then enter this into the settings you would expect it to use the gateway (10.10.20.1) on eth1 and thus be able to do recursive queries via vlan 20 image

But my testing suggest that it instead still uses the default gateway on eth0 (10.10.10.1) or simply fails to connect (maybe due to using a source IP not found on eth0) and all queries fail.

If I change the Docker network interfaces names around so a_eth0_10 becomes b_eth0_10 and b_eth0_20 becomes a_eth0_20 we change the order inside the container as well. So now:

Now the default gateway inside the container is 10.10.20.2 and now recursive queries will work.

If I remove the DNS Server IPv4 Source Addresses setting (and it resets to 0.0.0.0) the recursive queries will still work.

If I set the DNS Server IPv4 Source Addresses setting to 10.10.10.2 the recursive queries will stop working again.

ShreyasZare commented 4 months ago

@tannyl Thanks for the feedback. This feature was tested on Windows and Linux with native installation. It could be some issue with docker networking causing it to not bind to the specified interface. Will test this on my local setup.

Also, please check the DNS logs from the admin panel and post any error log you see regarding this here.

tannyl commented 4 months ago

@ShreyasZare I have done some more testing and have some log entries for you.

I made sure that the default gateway in the container did not allow recursive DNS queries (forcing the order of the interfaces inside the container as explained earlier) and set DNS Server IPv4 Source Addresses to the IP of the interface (and thus gateway/interface) it should use.

After a fresh restart one of the first errors is:

[2024-02-11 09:02:14 UTC] Error while updating root servers list by priming query: TechnitiumLibrary.Net.Dns.DnsClientNoResponseException: DnsClient failed to resolve the request '. NS IN': request timed out.
 ---> System.Net.Sockets.SocketException (110): Connection timed out
   at TechnitiumLibrary.Net.SocketExtensions.UdpQueryAsync(Socket socket, ArraySegment`1 request, ArraySegment`1 response, IPEndPoint remoteEP, Int32 timeout, Int32 retries, Boolean expBackoffTimeout, Func`2 isResponseValid, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\SocketExtensions.cs:line 143
   at TechnitiumLibrary.Net.Dns.ClientConnection.UdpClientConnection.QueryAsync(DnsDatagram request, Int32 timeout, Int32 retries, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\ClientConnection\UdpClientConnection.cs:line 271
   --- End of inner exception stack trace ---
   at TechnitiumLibrary.Net.Dns.ClientConnection.UdpClientConnection.QueryAsync(DnsDatagram request, Int32 timeout, Int32 retries, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\ClientConnection\UdpClientConnection.cs:line 280
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass76_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4166
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass76_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4344
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass76_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4081
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4396
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4384
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalNoDnssecResolveAsync(DnsDatagram request, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4457
   at TechnitiumLibrary.Net.Dns.DnsClient.UpdateRootServersAsync(NetProxy proxy, Boolean preferIPv6, Int32 retries, Int32 timeout, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 292
   at DnsServerCore.Dns.DnsServer.<StartAsync>b__168_1() in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 4549

After a short time a few more errors:

[2024-02-11 09:03:34 UTC] System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: A task was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at TechnitiumLibrary.Net.Http.Client.HttpClientNetworkHandler.InternalSendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Http\Client\HttpClientNetworkHandler.cs:line 99
   at TechnitiumLibrary.Net.Http.Client.HttpClientNetworkHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Http\Client\HttpClientNetworkHandler.cs:line 242
   at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at DnsServerCore.WebServiceAppsApi.GetStoreAppsJsonData(Boolean doRetry) in Z:\Technitium\Projects\DnsServer\DnsServerCore\WebServiceAppsApi.cs:line 183
   at DnsServerCore.WebServiceAppsApi.<StartAutomaticUpdate>b__11_0(Object state) in Z:\Technitium\Projects\DnsServer\DnsServerCore\WebServiceAppsApi.cs:line 96
[2024-02-11 09:03:47 UTC] [10.21.0.100:35472] Check for update was done {updateAvailable: False;}
System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: A task was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at TechnitiumLibrary.Net.Http.Client.HttpClientNetworkHandler.InternalSendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Http\Client\HttpClientNetworkHandler.cs:line 99
   at TechnitiumLibrary.Net.Http.Client.HttpClientNetworkHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Http\Client\HttpClientNetworkHandler.cs:line 242
   at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at DnsServerCore.WebServiceApi.GetCheckForUpdateJsonData() in Z:\Technitium\Projects\DnsServer\DnsServerCore\WebServiceApi.cs:line 78
   at DnsServerCore.WebServiceApi.CheckForUpdateAsync(HttpContext context) in Z:\Technitium\Projects\DnsServer\DnsServerCore\WebServiceApi.cs:line 102

Then I tried doing a recursive query: image

After a few mins this resulted in this error: image

Which came along with this log error:

[2024-02-11 09:05:44 UTC] TechnitiumLibrary.Net.Dns.DnsClientNoResponseException: DnsClient failed to recursively resolve the request 'dr.dk. A IN': no response from name servers [K.ROOT-SERVERS.NET (193.0.14.129), B.ROOT-SERVERS.NET (170.247.170.2), M.ROOT-SERVERS.NET (202.12.27.33), A.ROOT-SERVERS.NET (198.41.0.4), L.ROOT-SERVERS.NET (199.7.83.42), H.ROOT-SERVERS.NET (198.97.190.53), D.ROOT-SERVERS.NET (199.7.91.13), J.ROOT-SERVERS.NET (192.58.128.30), I.ROOT-SERVERS.NET (192.36.148.17), F.ROOT-SERVERS.NET (192.5.5.241), G.ROOT-SERVERS.NET (192.112.36.4), E.ROOT-SERVERS.NET (192.203.230.10), C.ROOT-SERVERS.NET (192.33.4.12)].
 ---> TechnitiumLibrary.Net.Dns.DnsClientNoResponseException: DnsClient failed to resolve the request 'dr.dk. A IN': request timed out.
 ---> System.Net.Sockets.SocketException (110): Connection timed out
   at TechnitiumLibrary.Net.SocketExtensions.UdpQueryAsync(Socket socket, ArraySegment`1 request, ArraySegment`1 response, IPEndPoint remoteEP, Int32 timeout, Int32 retries, Boolean expBackoffTimeout, Func`2 isResponseValid, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\SocketExtensions.cs:line 143
   at TechnitiumLibrary.Net.Dns.ClientConnection.UdpClientConnection.QueryAsync(DnsDatagram request, Int32 timeout, Int32 retries, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\ClientConnection\UdpClientConnection.cs:line 271
   --- End of inner exception stack trace ---
   at TechnitiumLibrary.Net.Dns.ClientConnection.UdpClientConnection.QueryAsync(DnsDatagram request, Int32 timeout, Int32 retries, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\ClientConnection\UdpClientConnection.cs:line 280
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass76_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4166
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass76_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4344
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass76_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4081
--- End of stack trace from previous location ---
   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 4443
   at TechnitiumLibrary.Net.Dns.DnsClient.RecursiveResolveAsync(DnsQuestionRecord question, IDnsCache cache, NetProxy proxy, Boolean preferIPv6, UInt16 udpPayloadSize, Boolean randomizeName, Boolean qnameMinimization, Boolean asyncNsRevalidation, Boolean dnssecValidation, NetworkAddress eDnsClientSubnet, Int32 retries, Int32 timeout, Int32 maxStackCount, Boolean cleanupResponse, Boolean asyncNsResolution, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 1080
   --- End of inner exception stack trace ---
   at TechnitiumLibrary.Net.Dns.DnsClient.RecursiveResolveAsync(DnsQuestionRecord question, IDnsCache cache, NetProxy proxy, Boolean preferIPv6, UInt16 udpPayloadSize, Boolean randomizeName, Boolean qnameMinimization, Boolean asyncNsRevalidation, Boolean dnssecValidation, NetworkAddress eDnsClientSubnet, Int32 retries, Int32 timeout, Int32 maxStackCount, Boolean cleanupResponse, Boolean asyncNsResolution, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 1762
   at DnsServerCore.WebServiceApi.ResolveQueryAsync(HttpContext context) in Z:\Technitium\Projects\DnsServer\DnsServerCore\WebServiceApi.cs:line 201
   at DnsServerCore.DnsWebService.WebServiceApiMiddleware(HttpContext context, RequestDelegate next) in Z:\Technitium\Projects\DnsServer\DnsServerCore\DnsWebService.cs:line 591
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.<Invoke>g__Awaited|10_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task)
ShreyasZare commented 4 months ago

@tannyl Thanks for the details. It seems that the UDP requests are being sent since there is no socket error otherwise. I would suggest that you run tcpdump on all interfaces in the container and see if outbound requests are being generated with correct source address. This should give a better view of the issue.

tannyl commented 4 months ago

@ShreyasZare I have done more testing after I got all the missing networking tools installed in the Docker image.

After looking at the results from tcpdump I could see that it does actually try to use the correct interface when doing the recursive dns queries. But it tries to find the DNS root servers via ARP requests: image

That lead me to believe that the problem might be a missing gateway on this interface: image

And there we have it. No gateway defined for this interface. Docker only defines 1 default gateway for the first interface inside the container.

If I add the gateway my self everything starts to work as expected (requires cap_add NET_ADMIN on the container): image

So the problem is due to how Docker networking works. I don't think you need to try to fix this problem, but I would suggest adding a check when saving the DNS Server IPvx Source Addresses and alert the user if they try to use an interface which does not have a gateway.

ShreyasZare commented 4 months ago

@tannyl Thanks for debugging the issue and posting the details here. This will help someone stuck with same issue.

alert the user if they try to use an interface which does not have a gateway

User may need to forward requests to another server in the same subnet so gateway is not really a hard requirement for this to work.