Open nasbdh9 opened 1 year ago
Do I understand correctly that the scenario you're describing is when you do get a response from the DNS, you just get an IP that's invalid in some way, yeah?
The error message you posted in the other thread suggest that the IP is invalid in that it's in some reserved IP range, not valid for the public internet. Is that right?
I don't think it's completely safe to assume that's an error. It's fine to run a tracker on a local network for instance.
Do I understand correctly that the scenario you're describing is when you do get a response from the DNS, you just get an IP that's invalid in some way, yeah?
Yes
The error message you posted in the other thread suggest that the IP is invalid in that it's in some reserved IP range, not valid for the public internet. Is that right?
No, this is not a reserved IP. It is just temporarily unavailable due to network failure or interference.
I don't think it's completely safe to assume that's an error. It's fine to run a tracker on a local network for instance.
Not local tracker
I see. So the DNS cache is fine. There is no need to invalidate it, the tracker just need to be retried (using the IP its name resolved to the first time).
Is that right?
using the IP its name resolved to the first time
not using the IP its name resolved to the first time, because he has made the tracker status unavailable (IP disturbed/faulty)
the tracker just need to be retried
Right, when the tracker status is unavailable let it re-resolve so that it can get another IP instead of continuing to try to report using the existing cache.
If the IP is available after re-resolution the tracker will resume working, if it is still unavailable then try to continue resolving. (it is better to provide a setting of X seconds to wait X seconds before resolving if it is still unavailable after X attempts, to avoid DNS Query Storm)
Ok. I was trying to distinguish between re-resolving and re-connecting.
A tracker that can't be connected to; I don't think it's safe to assume re-resolving would make a difference.
Typically, if a hostname has multiple IPs you receive multiple ones when resolving. If one fails, libtorrent will try the next IP.
Are you sure the problem is that the IP doesn't work and that you get a different IP when re-resolving?
Are you sure the problem is that the IP doesn't work and that you get a different IP when re-resolving?
yes, will be useful if DNS returns a different IP after re-resolving (tracker using CDN)
But after discussing it with other users, I think I may have complicated the issue. Back to the original idea, a "button" to immediately refresh the DNS cache would be enough. This even if system hosts are changed, the IP of the tracker can be updated immediately.
Some private trackers tend to put their announce behind multiple proxies (single domain name resolving to multiple ipv4 addresses). Sometimes some of these proxies temporarily go offline, and it is expected for the client to simply use another of the IPs. Do the recent DNS cache changes prevent this from working?
it is expected for the client to simply use another of the IPs. Do the recent DNS cache changes prevent this from working?
No, and the DNS cache isn't really involved in this logic. It just caches all the IPs a name resolves to.
Are you sure the problem is that the IP doesn't work and that you get a different IP when re-resolving?
yes, will be useful if DNS returns a different IP after re-resolving (tracker using CDN)
did you confirm that the first DNS lookup only returned a single IP address? did you confirm that the first IP you received was failing? how did you confirm it and what error message did you get? I would expect either "connection refused" or "host unreachable". did you confirm that a second DNS lookup gave you a different IP address? (was it still just a single IP?) Typically DNS servers would cache the response for some time, so you would presumably have to wait the TTL time before trying again. Does this hostname have a low TTL set?
Back to the original idea, a "button" to immediately refresh the DNS cache would be enough. This even if system hosts are changed, the IP of the tracker can be updated immediately.
I would want to understand what the problem is before making any changes.
did you confirm that the first DNS lookup only returned a single IP address?
No, multiple ip
did you confirm that the first IP you received was failing?
yes
how did you confirm it and what error message did you get? I would expect either "connection refused" or "host unreachable".
skipping tracker announce (unreachable)
did you confirm that a second DNS lookup gave you a different IP address? (was it still just a single IP?)
yes. even if dns does not return other ip, my can edit the system hosts to use other ip of cdn service provider But currently there is no way to refresh libtorrent DNS cache immediately after edit system hosts
Typically DNS servers would cache the response for some time, so you would presumably have to wait the TTL time before trying again. Does this hostname have a low TTL set?
Except for the system and program DNS caches, the DNS servers on the LAN do not have caching enabled, so each resolution on the LAN after the system and program caches expire will re-query the upstream public DNS servers
and it is expected for the client to simply use another of the IPs.
Nope.
That is not how standards work: when you get multiple "A" v4 IPs from one domain you need to perfectly randomly select one of them. You are not allowed to invalidate one of them. Now I don't quite remember what should happen if one of the IPs is bad and does not respond to TCP handshake, I wonder should you request DNS again... but I actually think it is just making the domain bad.
The whole balancing /round-robin standards are very complicated.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
There is an memory leak issue in qbittorrent which may be related to the dns caching and establishing tracker connection. As mentioned in https://github.com/qbittorrent/qBittorrent/issues/20117, my fake-ip on clash tun seems to be related in huge memory consumption in connection with trackers. Should i open another issue?
libtorrent version (or branch): all
platform/architecture: unrelated
compiler and compiler version: unrelated
prerequisite environment for problems to arise: tracker server uses CDN (such as Cloudflare) or tracker server domain with multiple IPs
happens when the problem arises: some ip returned by dns when resolving tracker servers are interfered with by the network, causing tracker to be unavailable
Current solutions to this problem: wait 1200 seconds (20 minutes, default) for the DNS cache to expire after the re-resolution to get another IP available and the tracker to resume normal operation
or still get an unavailable IP (interfered with, or unavailable itself) and then wait 1200 seconds and try again to resolve the new IP
Ideas for improving DNS cache: When the tracker status is unavailable, immediately discard the DNS cache for the wrong tracker, then re-resolve and try to re-report. (A publicly settable value may be needed to limit the number of retries, and to wait X seconds after X attempts to try again if it is always unavailable)
A time-limited blacklist can be set for erroneous IPs, and unavailable IPs will be in the list for a period of time (default might be 1 hour? or a value that can be set)