Cache DNS queries

[tomose]

qbittorrent-nox on linux

Seems like the client doesn't do any sort of caching of DNS queries at all.

Even if the caching was only for 30s at a time, it'd make for less noisy network activity.

[AGlezB]

Also on Windows please.

Just look at NextDNS analytics for the past 7 days while seeding some 140 small torrents.

This is for the last 30 days.

And that is just the top 6. I'm guessing qBittorrent alone ate twice over the 300K querys NextDNS intends to give for free.

[FranciscoPombal]

@arvidn what is libtorrent's behaviour regarding caching of DNS queries?

[arvidn]

by default it caches DNS lookups for 1200 seconds.

https://github.com/arvidn/libtorrent/blob/RC_1_2/src/resolver.cpp#L118

I would expect operating systems would cache for at least the domain TTL as well though.

[arvidn]

@AGlezB are those counting the actual DNS request messages sent to the server? Or are they counting calls to the OS function to request a lookup?

In either case, what are the TTLs for those names? Perhaps they are configured to be re-requested frequently.

[AGlezB]

@arvidn I honestly don't know. I have NextDNS Official App installed and the screenshots come from the Analitics tab in my.nextdns.io

[AGlezB]

Most of those are not resolving properly which makes a lot of sense.

@arvidn libtorrent has any logic to prevent repeating failed queries or maybe a delay between repeats?

[arvidn]

I recently added this

[tz1]

I too use NextDNS and saw a similar problem, but it is more subtle.

CACHING WILL NOT FIX THIS

NextDNS does NOT distinguish failed DNS requests from actually resolved ones.

The problem is the excess queries are a few seconds apart and happen when the lookup FAILS.

It seems to be instantly retried, often dozens of times without any backoff. There are several in my list, and when I manually check with nslookup, I don't get the "blocked", I get "doesn't exist", and from many DNS servers.

So if there is a failure looking up a tracker by DNS, it should mark it as not working and/or try maybe after a few seconds, then after a longer period (moving cables, router reboot), then timeout for another hour or so.

It also appears if there are several active torrents with the same unresolvable tracker, they will ALL try the DNS entry, so caching would help this as well since I often see multiple queries for the SAME tracker less than a second apart, and the TTLs can't be that small on the entry.

I was looking in the code, but didn't find where it is looking up the tracker DNS entry, and there is a lot of code.

[arvidn]

The host name lookups happen via a class libtorrent/resolver.hpp which does some caching. The underlying primitive of performing lookups goes through boost.asio's resolver though, which has each platform specific call to perform lookups.

Is it really the case that operating systems don't cache DNS requests? It would seem quite natural for them to do so, they even have the TTL.

[AGlezB]

Windows has a DNS Cache service but maybe libtorrent is bypassing it.

[tz1]

Operating systems generally do NOT cache requests unless the DNS "server" is running on the OS. Normally they simply forward requests upstream to that specified in the DHCP entry for DNS, typically the router. The router itself may or may not cache, and a problem is that most that forward to NextDNS (the program) do NOT cache replies, and the NextDNS app doesn't cache.

The other problem is nothing I know of caches the "doesn't exist" response, only the resolved IP address. So the large blob of tries for unresolavable domains will always result in attempts.

Maybe I need to look into libtorrent since it isn't in the tree.

[tz1]

to clarify, "gethostbyname()" doesn't cache. A local DNS server that forwards requests generally does when it gets a valid response (and for the TTL duraton).

[farid-fari]

@tz1 I don't think the issue about failing requests is the main issue at hand, it's probably a separate issue you can open. In my case, every tracker is resolved correctly, yet qBittorrent makes a DNS request per torrent. It makes them all nearly simultaneously, meaning that it's not waiting to cache results.

So I do think that caching would solve this, you just need to be clever about the way you do it -- maybe ping a tracker once and only once you've cached the DNS result make further requests to it.

This is a necessary feature though, every major piece of software has to go through this if it's doing this many DNS requests, it's incredibly wasteful as is right now.

Edit: my temporary fix is resolving these and putting them in /etc/hosts but clearly, not a long-term solution.

[crackwitz]

I'm running v4.3.0.1 on windows and I notice some CPU load from the "Dnscache" service of Windows (svchost), that goes away as soon as I close qB. it is not changed by enabling or disabling "Advanced > Resolve peer host names" so my suspicion is that it's related to trackers. I have ~200 "resumed" torrents, some of which surely have trackers that aren't there anymore.

I'd really like to get this CPU load taken care of. this wasn't an issue on qB v3.3 and whatever libtorrent version was used back then.

[nasbdh9]

dns returns an unavailable CDN IP, so it takes 20 minutes to refresh again? What needs to be set to disable these caches?

[arvidn]

I'm experimenting with this: https://github.com/arvidn/libtorrent/pull/7373