search

rakshasa / libtorrent

libTorrent BitTorrent library
http://rtorrent.net/downloads/
GNU General Public License v2.0
883 stars 210 forks source link

Simplify updating queue entries #248

Open kannibalox opened 7 months ago

kannibalox commented 7 months ago

The erase/insert pattern can be replaced by a single method that updates an existing entry's timer, or inserting if it doesn't exist.

This primary goal was to reduce CPU usage when handling incoming connections on large instances, but since the optimization was pretty generic it's been applied to many places.

stickz commented 4 months ago

Great Job @kannibalox. This pull request is 100% stable and greatly increases the number of peers rTorrent can process! It will allow over 10 gigabits to be pushed through rTorrent, provided the solid state hardware and system memory specs exist.

Is there any chance you could reduce CPU usage for downloading? I can't obtain any faster than 2 gigabits because it's CPU bound. I believe the problem is related to the way libtorrent rakshasa manages the pieces.

kannibalox commented 3 months ago

Is there any chance you could reduce CPU usage for downloading? I can't obtain any faster than 2 gigabits because it's CPU bound. I believe the problem is related to the way libtorrent rakshasa manages the pieces.

I did find a couple other things that could be optimized, but without a profile of actual usage from perf record or something similar, it's hard to know where the busiest parts are, and I didn't want to flood the repo with lots of micro-optimization PRs

stickz commented 3 months ago

I did find a couple other things that could be optimized, but without a profile of actual usage from perf record or something similar, it's hard to know where the busiest parts are, and I didn't want to flood the repo with lots of micro-optimization PRs

@kannibalox Please send me an email. I have what you need. I don't want to post sensitive data here. I recorded 100% CPU usage at 4.3ghz downloading at 2gbits.

stickz commented 3 months ago

@kannibalox Thank you for your email. I have determined the download delegator which decides the pieces to queue is responsible for the high CPU usage. There are multiple std::find_if search patterns being called very rapidly at 2gbps. https://github.com/rakshasa/libtorrent/blob/master/src/download/delegator.cc