search

rakshasa / rtorrent

rTorrent BitTorrent client
https://github.com/rakshasa/rtorrent/wiki
GNU General Public License v2.0
4.08k stars 408 forks source link

Launch is extremely slow and rtorrent is completely blocked with thousands of multitracker torrents #1044

Open PulsedMedia-Aleksi opened 3 years ago

PulsedMedia-Aleksi commented 3 years ago

During launch of rTorrent instance with a lot of public tracker torrents the launch takes forever. CPU pegged at "100%" (ie 1 core maxed out) as a avalanche of dns requests and tracker updates start.

This can take 10minutes or so before rTorrent to response in neither console or xmlrpc

Checking logs it's a huge amount of (failed) dns resolutions and tracker updates.

Debian 10 0.9.8 / 0.13.8-udns

I would hazard a guess that #999 is related to this -- rTorrent is unable to respond to even the simplest things, you cannot then expect it to perform well.

PulsedMedia-Aleksi commented 3 years ago

high number of http max open makes this even slower, so relates to #1040

PulsedMedia-Aleksi commented 3 years ago

Tried on "0.9.7" and that has cli stuck BUT xmlrpc is immediately responsive.

stickz commented 3 years ago

If the CPU usage is pegged at 100% on launch, rakshasa/libtorrent may not be built properly with udns. This should reach 200% for approximately half the time using 2 full cores. Once the asynchronous dns queries are done, it will drop back down to 100%.

I would also recommend a local dns server and disabling the built in dns cache for rTorrent to improve performance. This work around will move a bunch of load off the bottlenecked main thread and onto anther process like dnsmasq for example.

pyroscope commented 3 years ago

Also c-ares or not is not specified. Some reliable numbers on how this and e.g. the presence of a local DNS cache influences startup times would make judging ROI on possible code improvements way more substantiated.

stickz commented 3 years ago

This problem also happens with only UDP trackers. In this case, it's not related to if curl is installed with c-ares support. My local DNS cache is hit pretty heavily with requests at start up. This appears to be related to the DNS queries.

I think the solution to this problem would be to update trackers in an asynchronous way on a separate thread, so the main thread is not blocked during start up. A feature where users can disable certain failed trackers from loading on all torrents, at the very least would be helpful. Like tracker.disable.list.add = "udp://tracker.coppersurfer.tk:6969/announce" for instance.