Settings for seeding thousands of smaller torrents?

[GoogleCodeExporter]

I'm seeding thousands of torrents, and its choking my connection so much that 
the net becomes inaccessible. At the same time, I still need these torrents to 
seed when a peer requests them.

I've gone through many variations of the settings here, 
http://www.libtorrent.org/tuning.html, but have a few questions.

Here's what hasn't worked so far:
- Changing the active_limit, and active_seeds. They could be small or large, 
but it still chokes my connection. I am unsure what I should have those numbers 
at also, the automanaged uploads are still unclear to me.

- reducing the upload_rate_limit. My connection is still choked.

- Using settings.new_defaults, it chokes up my RAM and connection. Using 
settings.min_memory_usage saves my RAM and my connection, but doesn't seed 
anything, which is equally useless.

A possibly related question: Is there any network activity done repeatedly that 
I could slow down that might be choking my connection, such as announcing 
thousands of torrents to multiple trackers repeatedly?

Does anyone have good working settings for seeding thousands of torrents 
effectively?

Thanks in advance.

Original issue reported on code.google.com by Tchouli...@gmail.com on 25 Jun 2015 at 3:14

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I turned on a bunch of alerts to see what was clogging up the network, and I'm 
finding that my logs are filled with tracker timeouts:

(http://mgtracker.org:2710/announce) (-1) Connection timed out "" (0)
...

And I think libtorrent might be repeatedly retrying them, and that's whats 
choking the connection.

Also, all my torrents list all the public trackers here: 
https://gist.github.com/mcandre/eab4166938ed4205bef4

Should they only list 1 or 2 public trackers instead?

Original comment by Tchouli...@gmail.com on 25 Jun 2015 at 4:38

[GoogleCodeExporter]

The first thing I think you should do is to understand exactly what you mean by 
"choking the connection". Things you may want to consider to do this would be 
to:

1. wireshark your connection to see if there's any specific traffic that 
correlates well with your connection being choked.
2. profile libtorrent to see where it's spending all that CPU

"ghost torrents" is a feature in master where torrents can be unloaded from 
RAM, and requested to be loaded back in again when needed. It's a way to save 
RAM.

If each of your torrent announces to a lot of trackers, you may want to limit 
that. either by removing trackers from the torrents before you add them (keep 
in mind that UDP trackers are cheaper) and/or lowering your tracker_limit 
setting (i.e. the number of torrents that are allowed to announce to their 
tracker). But, understanding what the problem is should be the highest priority.

Original comment by arvid.no...@gmail.com on 26 Jun 2015 at 4:15

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I just tried setting the tracker limit:
sessionSettings.setActiveTrackerLimit(1);

But this seems to do nothing when used with the settings above. As it goes 
through my library adding torrents, it still tries to do tracker announces for 
each of them.

I did a wireshark analysis, but I'm not sure what I was supposed to gather from 
it. The UDP traffic to all kinds of ports and peers went up significantly, but 
I already knew that.

Do you have any examples of what the settings should be, for potentially 
seeding thousands of torrents?

The new_defaults isn't made for this, and me tuning the session_settings is 
getting to be about a page long of unverified fixes.

Original comment by Tchouli...@gmail.com on 26 Jun 2015 at 5:19

[GoogleCodeExporter]

Okay, it turns out this line was the one choking my connection:

  sessionSettings.setActiveDHTLimit(-1);

I changed it to 5, and my server seems to be doing fine. I still would 
definitely enjoy some 'seeding thousands of torrents' settings though.

Original comment by Tchouli...@gmail.com on 27 Jun 2015 at 1:03

[GoogleCodeExporter]

that's not the default setting, is it?
did you try _just_ the default settings?

Original comment by arvid.no...@gmail.com on 27 Jun 2015 at 1:46

[GoogleCodeExporter]

I did, but I had problems with active limits. Currently I have about 500 
torrents, but I'm doing some larger-scale testing today to ramp that up. I'll 
let you know if I run into any problems.

Original comment by Tchouli...@gmail.com on 27 Jun 2015 at 1:23