Support DNS over TCP

[mzpqnxow]

Hey there @blechschmidt, I generally use massdns over UDP as it makes more sense (much less overhead) but recently I had a case where I was stuck using TCP, due to an unusual connectivity scenario that broke UDP DNS because I was losing the source port- it wasn't a NAT type scenario, it was over a port forward.

I realize that there is a performance penalty with TCP, but it seems like something reasonable to support and I think that while massdns is certainly high-performance, there wouldn't be a severe penalty. Additionally, this would remain as a non-default option

While I would like to write this myself and send a PR-

1. I'm not sure if you're interested in accepting it
2. My time is very limited right now (though I understand yours probably is as well)

What are your thoughts on this? Do you agree TCP support would be a reasonable feature to have?

[blechschmidt]

If implemented cleanly (minimized statefulness, minimum number of memory allocations required), I will accept such a PR.

There are basically two options coming to my mind:

1. The use of normal TCP sockets could work but it could have some drawbacks, such as exceeding the open file limits or using up all available ports if the limits are increased. TCP_NODELAY could be used in order to send one segment per DNS question.
2. The implementation of a custom TCP stack using raw sockets could be considered and I assume that it would be the best solution in terms of achievable performance. The stack could be relatively dumb as one DNS questions fits into a TCP segment but one could think of parallelizing the use of a single source port.

If not integrated directly into massdns, another possibility consists in implementing a translator application that is independent from massdns, serves on one or multiple local UDP ports and redirects incoming requests to the DNS servers via TCP accordingly.

All in all, I don't consider this to be of high priority and I probably will not implement this in the near future.

[mzpqnxow]

@blechschmidt thanks for the response.

The raw sockets approach was what I had in mind.

Can we keep this open for now tagged as you prefer so that I have it in my queue for when I have the time to work on this?

Thanks again, I know time is at a premium for everyone and it's difficult to allocate time to respond to feature requests that are certainly not a priority for the majority of users

[blechschmidt]

The tcp-simple branch now implements the standard socket approach. The performance is bad, as expected, but I needed a quick solution for truncated DNS responses, which I could hack together in a few hours (in contrast to the raw sockets approach, which I also began to implement). --tcp-enabled enables the TCP option in case the response is truncated, while --tcp-only will instruct MassDNS to always use TCP.

Use a low -s value and set the file descriptor limit using ulimit. The code on that branch is not well-tested and is likely to be buggy.

[mzpqnxow]

ah, great- thank you for putting the time into it, it's appreciated

Hopefully I'll have some time to test and open issues or PRs where applicable

I assume the raw socket + user-space (partial) TCP/IP stack version will be completed next week? (joking, obviously ...)

I'm not necessarily expecting you to ever have the time to put into that, but if you do plan to, you may find masscan to be a somewhat useful reference. If you're not already familiar, it has a limited user-space TCP/IP stack with similar goals (not a complete implementation, just stateful enough to grab various protocol banners, do basic negotiation, etc.)

Based on how clean and disciplined your coding style is, I don't expect the masscan code would ever be fork-liftable into massdns. I can say though that it works and has proven over time to be solid in practice. It's just a bit of a mess in many places. Aside from being partially hacked together from other sources, it's also a bit complicated in places for the sake of being able to reach really ridiculous transmission rates- more than is appropriate for massdns, because nobody wants to hammer a resolver that hard

[blechschmidt]

I assume the raw socket + user-space (partial) TCP/IP stack version will be completed next week? (joking, obviously ...)

I was so unsatisfied with the performance of the standard sockets approach (less than 1,000 qps) that the user stack approach is already mostly finished, while the performance is promising. I will probably release it in December.

Based on how clean and disciplined your coding style is

Let's be real here. Currently, main.c is one huge mess and needs some serious refactoring.

[blechschmidt]

You can now use --tcp-raw. For this to work, massdns needs to be started with the appropriate capabilities (with CAP_NET_RAW). Additionally, when using raw mode, matching TCP packets must be dropped such that the local machine does not generate TCP reset packets for the TCP connections which it is not aware of (e.g. using iptables -I INPUT -p tcp --sport 53 -j DROP). Please don't expect these --tcp-* arguments to be stable though.