SOCKS5 proxy doesn't work

I have a Dante SOCKS5 proxy server running on a VPS with Debian 9. This proxy works fine with any client I've tested (Firefox and cURL, for example), but for some reason it doesn't work with Technitium DNS Server (TDS) v6.2.1 (I'm using TDS under Windows 10 Pro 64 bits).

To test TDS I first flush its cache, and then I query the A record for google.com from the DNS client tab of TDS web GUI. This is what I get using my SOCKS5 proxy:

{
  "Metadata": {
    "NameServer": "my_pc:53 (127.0.0.1:53)",
    "Protocol": "Udp",
    "DatagramSize": "28 bytes",
    "RoundTripTime": "2744,49 ms"
  },
  "Identifier": 14494,
  "IsResponse": true,
  "OPCODE": "StandardQuery",
  "AuthoritativeAnswer": false,
  "Truncation": false,
  "RecursionDesired": true,
  "RecursionAvailable": true,
  "Z": 0,
  "AuthenticData": false,
  "CheckingDisabled": false,
  "RCODE": "ServerFailure",
  "QDCOUNT": 1,
  "ANCOUNT": 0,
  "NSCOUNT": 0,
  "ARCOUNT": 0,
  "Question": [
    {
      "Name": "google.com",
      "Type": "A",
      "Class": "IN"
    }
  ],
  "Answer": [],
  "Authority": [],
  "Additional": []
}

As you can see, the Answer is completely empty.

If I look at the TDS logs, I get these results:

[2021-04-17 22:07:59 UTC] DNS Server recursive resolution failed for QNAME: google.com; QTYPE: A; QCLASS: IN;

TechnitiumLibrary.Net.Dns.DnsClientException: DnsClient recursive resolution failed: no response from name servers [d.gtld-servers.net:53 (192.31.80.30:53), m.gtld-servers.net:53 (192.55.83.30:53), k.gtld-servers.net:53 (192.52.178.30:53), h.gtld-servers.net:53 (192.54.112.30:53), f.gtld-servers.net:53 (192.35.51.30:53), i.gtld-servers.net:53 (192.43.172.30:53), l.gtld-servers.net:53 (192.41.162.30:53), c.gtld-servers.net:53 (192.26.92.30:53), e.gtld-servers.net:53 (192.12.94.30:53), b.gtld-servers.net:53 (192.33.14.30:53), a.gtld-servers.net:53 (192.5.6.30:53), j.gtld-servers.net:53 (192.48.79.30:53), g.gtld-servers.net:53 (192.42.93.30:53)]

 ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.

   at TechnitiumLibrary.IO.StreamExtension.ReadBytesAsync(Stream s, Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.IO\StreamExtension.cs:line 53

   at TechnitiumLibrary.Net.Proxy.SocksProxyReply.ReadReplyAsync(Stream s) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Proxy\SocksProxyReply.cs:line 85

   at TechnitiumLibrary.Net.Proxy.SocksProxy.RequestAsync(Stream s, SocksProxyRequest request) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Proxy\SocksProxy.cs:line 121

   at TechnitiumLibrary.Net.Proxy.SocksProxy.UdpAssociateAsync(EndPoint localEP) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Proxy\SocksProxy.cs:line 301

   at TechnitiumLibrary.Net.Proxy.SocksProxy.IsUdpAvailableAsync() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Proxy\SocksProxy.cs:line 178

   at TechnitiumLibrary.Net.Dns.DnsClient.<>c__DisplayClass38_0.<<InternalResolveAsync>g__DoResolveAsync|1>d.MoveNext() in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 1501

--- End of stack trace from previous location ---

   at TechnitiumLibrary.Net.Dns.DnsClient.InternalResolveAsync(DnsDatagram request, Boolean qnameMinimization) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 1728

   at TechnitiumLibrary.Net.Dns.DnsClient.RecursiveResolveAsync(DnsQuestionRecord question, IDnsCache cache, NetProxy proxy, Boolean preferIPv6, Boolean randomizeName, Boolean qnameMinimization, Int32 retries, Int32 timeout, Int32 maxStackCount) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 570

   --- End of inner exception stack trace ---

   at TechnitiumLibrary.Net.Dns.DnsClient.RecursiveResolveAsync(DnsQuestionRecord question, IDnsCache cache, NetProxy proxy, Boolean preferIPv6, Boolean randomizeName, Boolean qnameMinimization, Int32 retries, Int32 timeout, Int32 maxStackCount) in Z:\Technitium\Projects\TechnitiumLibrary\TechnitiumLibrary.Net\Dns\DnsClient.cs:line 871

   at DnsServerCore.Dns.DnsServer.RecursiveResolveAsync(DnsDatagram request, IReadOnlyList`1 viaForwarders, Boolean cachePrefetchOperation, Boolean cacheRefreshOperation, TaskCompletionSource`1 taskCompletionSource) in Z:\Technitium\Projects\DnsServer\DnsServerCore\Dns\DnsServer.cs:line 1804

However, if I download, install, and run Tor Browser, and use its SOCKS5 proxy from TDS, then TDS works perfectly fine.

As the Tor SOCKS5 proxy works fine, at first glance it may appears like the problem is related to my custom SOCKS5 proxy, but then, why does my SOCKS5 proxy work properly with Firefox (among others)? Why does it fail only with TDS?

If I look to my SOCKS5 proxy logs I can see these errors when I query my TDS server:

Apr 18 00:33:57 (1618698837.365366) danted[6889]: info: pass(1): tcp/accept [: 10.0.2.2.55154 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.369473) danted[6890]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.369965) danted[5465]: warning: EOF from request-child 6890
Apr 18 00:33:57 (1618698837.370006) danted[5465]: warning: sigchld(): request-child 6890 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.372829) danted[6889]: info: pass(1): tcp/accept [: 10.0.2.2.55155 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.374662) danted[6891]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.375040) danted[5465]: warning: EOF from request-child 6891
Apr 18 00:33:57 (1618698837.375083) danted[5465]: warning: sigchld(): request-child 6891 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.377601) danted[6889]: info: pass(1): tcp/accept [: 10.0.2.2.55156 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.379685) danted[6892]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.381072) danted[5465]: warning: EOF from request-child 6892
Apr 18 00:33:57 (1618698837.381135) danted[5465]: warning: sigchld(): request-child 6892 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.384028) danted[6889]: info: pass(1): tcp/accept [: 10.0.2.2.55157 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.385800) danted[6893]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.386171) danted[5465]: warning: EOF from request-child 6893
Apr 18 00:33:57 (1618698837.386434) danted[5465]: warning: sigchld(): request-child 6893 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.393331) danted[6889]: info: pass(1): tcp/accept [: 10.0.2.2.55158 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.398474) danted[6894]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.399495) danted[5465]: warning: EOF from request-child 6894
Apr 18 00:33:57 (1618698837.399541) danted[5465]: warning: sigchld(): request-child 6894 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.402614) danted[6889]: info: pass(1): tcp/accept [: 10.0.2.2.55159 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.411425) danted[6895]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.415716) danted[5465]: warning: EOF from request-child 6895
Apr 18 00:33:57 (1618698837.415776) danted[5465]: warning: sigchld(): request-child 6895 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.419018) danted[6889]: info: pass(1): tcp/accept [: 10.0.2.2.55160 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.421162) danted[6897]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.422237) danted[5465]: warning: sigchld(): request-child 6897 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.422249) danted[5465]: error: unexpecteddeath(): 10 child deaths in 0 seconds.  Not something we are expecting so possibly something is wrong.  Locking childcount for a while (10 seconds), hoping things will stabilize
Apr 18 00:33:57 (1618698837.422254) danted[5465]: info: disable_childcreate(): disabling creation of new child processes: large amount of processes suddenly died (no system error)
Apr 18 00:33:57 (1618698837.422622) danted[5465]: warning: need to add a new child process to handle client load, but unable to do so at the moment: large amount of processes suddenly died (no system error)
Apr 18 00:33:57 (1618698837.427765) danted[7451]: info: pass(1): tcp/accept [: 10.0.2.2.55161 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.430882) danted[6898]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.431289) danted[5465]: warning: sigchld(): request-child 6898 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.446150) danted[7451]: info: pass(1): tcp/accept [: 10.0.2.2.55162 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.448489) danted[6900]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.449279) danted[5465]: warning: sigchld(): request-child 6900 exited unexpectedly on signal SIGABRT
Apr 18 00:33:57 (1618698837.452759) danted[7451]: info: pass(1): tcp/accept [: 10.0.2.2.55163 0.0.0.0.2408
Apr 18 00:33:57 (1618698837.455045) danted[6901]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15
2014/08/24 17:43:41 michaels Exp $.  Please report this to Inferno Nettverk A/S at "dante-bugs@inet.no".  Please check for a coredump too.
Apr 18 00:33:57 (1618698837.455857) danted[5465]: warning: sigchld(): request-child 6901 exited unexpectedly on signal SIGABRT
Apr 18 00:34:07 (1618698847.423183) danted[5465]: info: enable_childcreate(): creation of new child processes enabled again

Please, let me remark that those error messages only appear when I use my SOCKS5 proxy from TDS, because when I use it from another program (Firefox, etc.) I get no errors at Dante's logs.

I've also tried to use PuTTY, creating a dynamic tunnel (which acts as a SOCKS5 proxy too) when connecting to my VPS, but I get the same results: it doesn't work with TDS, but works fine with other tools.

May there be a bug in the SOCKS5 protocol implementation that TDS is using? According to TDS and Dante's log, it looks like the socket is closed unexpectedly for both parties, as TDS says:

---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.

while Dante shows:

Apr 18 00:33:57 (1618698837.398474) danted[6894]: warning: an internal error was detected at sockd_request.c:2949, value 0, expression "IPADDRISBOUND(&io.src.laddr)".  Version: $Id: sockd_request.c,v 1.849.4.15

How can I help to debug this issue?

Thanks.

Just a quick update.

I've checked that TDS can download a blocklist from a URL when using my SOCKS5 proxy. These are the logs (with proxy on):

[2021-04-17 23:52:22 UTC] [127.0.0.1:58088] [admin] Block list update was triggered.
[2021-04-17 23:52:28 UTC] DNS Server successfully downloaded block list (2,2 MB): https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
[2021-04-17 23:52:28 UTC] DNS Server is reading block list from: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
[2021-04-17 23:52:28 UTC] DNS Server block list file was read (76473 domains) from: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
[2021-04-17 23:52:28 UTC] DNS Server block list zone was loaded successfully.
[2021-04-17 23:52:28 UTC] DNS Server config file was saved: C:\Program Files (x86)\Technitium\DNS Server\config\dns.config

If I disable my proxy service, then TDS cannot download anything, so it's actually using the SOCKS5 proxy to download that URL.

I hope it helps to give some debugging hints, although I'm even more confused. 😅

Thanks for the detailed feedback. I will test this once and let you know the results.

I just installed Dante proxy on Ubuntu 20.04 and its working well. The DNS Server is able to do recursive resolution through the socks5 proxy without any errors. Also tested with all forwarder protocol combinations too and its all working well.

I even checked all the network traffic using Wireshark and things are as expected.

If possible, do share your danted.conf file so that I can try the exact config and test again.

Thank you @ShreyasZare for your quick response, and detailed investigation.

This is the content of my /etc/danted.conf:

logoutput: /var/log/socks.log
internal: 0.0.0.0 port = 2408
external: eth0
socksmethod: none
user.privileged: root
user.notprivileged: nobody

client pass {
 from: 0.0.0.0/0 to: 0.0.0.0/0
 log: error connect disconnect
}

client block {
 from: 0.0.0.0/0 to: 0.0.0.0/0
 log: connect error
}

socks pass {
 from: 0.0.0.0/0 to: 0.0.0.0/0
 log: error connect disconnect
}

socks block {
 from: 0.0.0.0/0 to: 0.0.0.0/0
 log: connect error
}

I'm afraid this is some kind of esoteric network problem, very specifically related to my setup, because of the following reasons:

TDS can download a HTTPS resource with my Dante installation (because it can update its blocklist) but cannot forward a DNS query, so the issue is (network) protocol related.
I've created another Debian instance on my VPS, installed and configured a SSH server on it, and then created a dynamic tunnel with PuTTY form the same machine that runs TDS. In this scenario TDS can forward DNS queries too, which proofs that is not a general problem on my VPS.
I've created a Debian 9 virtual machine using VirtualBox (also on the same machine that runs TDS), replicated my VPS setup on it, and it shows the same behaviour as my VPS: TDS can download a HTTPS resource through my Dante running on VirtualBox, but cannot forward a DNS query.

I'd like to debug this, but I still don't know how. Is there a way to make TDS logs more verbose?

Thanks for the config. I tried the same on Ubuntu 20.04 and its working well here again.

As you mentioned that the DNS server is able to download HTTPS resources and also your web browser is working with it, so the proxy is working well for TCP requests. But, for UDP requests, it seems to be failing due to some local issue.

This issue will affect only recursive resolution and thus if you are planning to use a forwarder then it would work with the proxy for other transport protocols.

The error stack trace is the max debug log that is possible on the DNS server. Do try to capture the traffic on the network with tcpdump and share the pcap file. This will make is much clear as to what is going on at the network level.

I think I've partially track down the issue, and it seems to be related to my VPN provider (to simplify my examples I omitted that I'm using a VPN on my VPS, and connecting to it with OpenVPN). Using this specific VPN I can repeat the issue on any VM following these steps:

Install Debian 9 in a VM.
Install ssh, resolvconf, and openvpn on that VM.
On the VM, connect to this VPN provider with OpenVPN.
Connect to your VM using PuTTY, creating a dynamic tunnel in that connection.
Configure TDS to use that tunnel as a SOCKS5 proxy.
Leave the Forwarders area empty.
Select DNS-over-UDP (default) as the Forwarder Protocol.

This way, the TDS client requests don't work. However, if you enter a DNS server on the forwarders area, and apply the settings, then the TDS client works as expected (even with DNS-over-UDP). So TDS needs to know the DNS server (IP or FQDN) in order to work with my VPN using DNS-over-UDP.

Interestingly enough, if I use another VPN provider (I've tested a free account on ProtonVPN), then I don't need to specify the forwarders. So the issue only appears when I'm using my current VPN provider (which I prefer not to disclose for privacy reasons) and leave the forwarders area empty.

This weekend I'll try to capture the network traffic on my VM as you suggest, with and without filling the forwarders area, and tell you the results.

Thank you for your patience.

Ahh ... openvpn!

So, openvpn by default hijacks all UDP port 53 traffic and sends it over the VPN tunnel when connected. This is a feature called block-outside-dns to prevent DNS queries from leaking directly to your Internet which would be devastating for privacy/security reasons.

There is an option which you can use in your vpn client config file to stop this default behavior so that you can query to a DNS server that is configured. To disable the feature, just add the following line in your openvpn client config file:

pull-filter ignore "block-outside-dns"

Make sure to do a DNS leak test if you want to protect your privacy over VPN with this option disabled.

Thank you for your suggestion. I've tried to add that line to my OpenVPN config file, but it doesn't change the results.

However, I've discovered that I can type almost any IP address (even bogons or broadcast) on the forwarders area on TDS, and it will work using OpenVPN + SOCKS5 + DNS-over-UDP on the setup previously described. For example, I've tried some random IP addresses (like 1.0.0.0, 10.0.0.0, or 24.25.26.27), and all of them work fine as long as the forwarders area is not empty.

On my VPS hosting happens something similar, but I need to use DNS-over-TCP (TDS needs its forwarders area not to be empty in order to work with OpenVPN + SOCKS5 + DNS-over-TCP). Once more, it seems to accept almost any IP address, as long as it's not empty anything will work.

Thank you for your suggestion. I've tried to add that line to my OpenVPN config file, but it doesn't change the results.

However, I've discovered that I can type almost any IP address (even bogons or broadcast) on the forwarders area on TDS, and it will work using OpenVPN + SOCKS5 + DNS-over-UDP on the setup previously described. For example, I've tried some random IP addresses (like 1.0.0.0, 10.0.0.0, or 24.25.26.27), and all of them work fine as long as the forwarders area is not empty.

Since you have SOCKS5 running, any IP address that you put in forwarder will be sent to SOCKS5 proxy and the proxy will forward it further. So, now if you have openvpn on the server running on the same server, then the DNS request will get hijacked by openvpn and in such case the destination IP address doesn't matter.

On my VPS hosting happens something similar, but I need to use DNS-over-TCP (TDS needs its forwarders area not to be empty in order to work with OpenVPN + SOCKS5 + DNS-over-TCP). Once more, it seems to accept almost any IP address, as long as it's not empty anything will work.

Without the forwarders configured, the DNS server will do recursive resolution where it has validation checks that could reject the hijacked responses making it non functional.

Well,

After some more testing, I've found the following:

My VPN provider is redirecting all DNS traffic (over UDP and TCP) to their own DNS servers, no matter what (a bit extreme measure, but I think they are trying to prevent DNS leaks). That's why, when I enter a random (even non-routable) IP address on the forwarders, TDS always works, because in reality my VPN DNS servers will intercept and reply to those queries.
My ISP is actively blocking some DNS traffic. For example, I cannot send queries to 1.1.1.1 using DNS-over-UDP, neither DNS-over-TCP. Until I've realized, it has been giving me some headaches, and I find no good reason for this behavior, but at least is good to know.
Whatever is blocking DNS-over-UDP requests on my original setup, doesn't block them on my other instances or VMs, so I'll have to either use DNS-over-TCP or redo my setup.

Thank you very much for you time and patience @ShreyasZare.

Thanks for the detailed analysis. This would help someone searching for similar issues for sure.

TechnitiumSoftware / DnsServer

SOCKS5 proxy doesn't work #255