search

rakshasa / rtorrent

rTorrent BitTorrent client
https://github.com/rakshasa/rtorrent/wiki
GNU General Public License v2.0
4.05k stars 412 forks source link

rtorrent starts up ok then eventually goes to 100% cpu usage and spits out failed to connect and connection to all trackers halt. #1208

Open evilbulgarian opened 1 year ago

evilbulgarian commented 1 year ago

I have about 2100 torrents, all new torrents seem to not even start when added via flexget. I am attaching my config and emerge --info. I have tried recompiling curl with nss only support same issue.

gypsyops_emerge.info.txt rtorrent.rc.txt

1675560638 I 55565E134F5D55B1E4B6372D709E61046211D0E9->tracker_list: failed to connect to tracker ...
1675561079 I handshake_manager->108.50.139.230: Received error: message:7 network error.
1675561151 I handshake_manager->75.82.54.146: Received error: message:7 network unreachable.
evilbulgarian commented 1 year ago 
❯ curl --version
curl 7.87.0 (x86_64-pc-linux-gnu) libcurl/7.87.0 NSS/3.79.2 zlib/1.2.13 zstd/1.5.2 c-ares/1.18.1 nghttp2/1.51.0
Release-Date: 2022-12-21
Protocols: dict file ftp ftps http https imap imaps mqtt pop3 pop3s rtsp smtp smtps tftp
Features: AsynchDNS HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM SSL threadsafe UnixSockets zstd
kannibalox commented 1 year ago

Did rTorrent work for you built with earlier versions of curl, or has it been behaving like this all the time?

evilbulgarian commented 1 year ago

I have 2 other servers and they seem to be ok with the curl compiled with openssl. Going to set up debug logging on them and check for same errors. It something that happens over time it seems.

JohnFlowerful commented 1 year ago

It was fine with at least curl 7.84.0. This version (amongst others) has been removed from package managers. Maybe a curl regression similar to #580

New torrents start working eventually for me though.

Edit: I installed net-misc/curl-7.86.0-r3 and can confirm that this resolved the issue for me.

evilbulgarian commented 1 year ago

It was fine with at least curl 7.84.0. This version (amongst others) has been removed from package managers. Maybe a curl regression similar to #580

New torrents start working eventually for me though.

Edit: I installed net-misc/curl-7.86.0-r3 and can confirm that this resolved the issue for me.

I downgraded to same version still startup is fine then goes to 100% cpu with same network error messages :( thanks for suggestion tho.

evilbulgarian commented 1 year ago

OK, I think that the reverted curl actually is making my load avg a lot better its not constantly at 100% cpu only once in a while. Do you think we should submit a bug in gentoo as well @JohnFlowerful?

JohnFlowerful commented 1 year ago

My issue is fixed with now in-tree net-misc/curl-7.88.0-r1. I'm not sure of the cause, but curl's issue tracker had a few people reporting inconsistencies between 7.86 and 7.87. I'd be guessing, but https://github.com/curl/curl/pull/10160 looks relevant.

I don't think it's a Gentoo issue. If it's still an issue with the new version, you're better off reporting directly to curl.

stickz commented 1 year ago

Does this pull request help your issue? Your load average will be significantly lower. It will just take a little bit more time for trackers to update initially. https://github.com/rakshasa/rtorrent/pull/1131

tholin commented 1 year ago

I might be experiencing the same problem. Rtorrent sometimes get stuck using 100% cpu for several seconds or even minutes at a time. I looked into the problem and it turns out to be a regression in Gentoo's net-misc/curl.

The ebuild for curl 7.84.0 and before had a default disabled "threads" useflag but more recent version removed the useflag and defaults to --enable-pthreads and --enable-threaded-resolver. This is the change that cause rtorrent to use 100% cpu. If I build curl-7.87.0-r2 with USE="-adns" and modify the ebuild to disable threaded-resolver the problem goes away.

Rtorrent use 100% cpu because it's stuck calling the epoll system call in a busy loop. Without a multithreaded resolver I assume rtorrent would just block during name resolves. With asynchronous resolves rtorrent keeps executing and goes wild.

Both evilbulgarian and JohnFlowerful seems to be Gentoo users so I'm guessing it's the same problem. All Gentoo versions of curl after 7.84.0 after broken for me since they all force --enable-threaded-resolver. The problem only triggers when DNS resolves are a bit flakey and have a risk to timeout.