multi tracker specification

[arvidn]

I don't believe anyone has ever implemented the behavior specified by the multi-tracker extension, BEP12.

Specifically, uTorrent found it more useful to treat separate tiers as independent sources of peers, and will announce to trackers in separate tiers in parallel. Everyone seemed to expect and rely on this behavior so it was also implemented in libtorrent.

I propose the specification be updated to match the de-facto standard.

Now there's a private tracker that uses the multi-tracker extension in its torrents, but puts all trackers in the same tier. My reading of the specification (when I first implemented it in libtorrent) was that trackers in the same tier should be treated as load-balancing front-ends to the same peer database. Announces should be directed to a random host in that group, to evenly distribute the load.

I have not tested this, but this forum post suggests that uTorrent either doesn't support multiple trackers in the same tier, or it pins a single tracker to be used consistently within the tier (probably the latter).

The specification is not very clear on this point. We should clarify this as well.

Another aspect the specification is silent on is how &event=started and &event=stopped should be treated. Currently, libtorrent records whether it has sent a started event per tracker, regardless of tier. So this is not quite treating every tracker in a tier as just front-ends to the same database, since it will send started to the same tier multiple times, via different trackers.

Any thoughts on what the behavior should be?

[atomashpolskiy]

In fact there's at least one implementation that does exactly what's described in BEP-12 :) Though it does not disconnect from current peers upon switching between tiers, and I'm not sure if this behavior is consistent with BEP-27... as is announcing to multiple tiers in parallel, which means that the peer databases of different tiers become mixed with each other and ultimately converge to the same superset of peers (not to say that this is a bad thing). So yeah, there is definitely some confusion with multi-trackers and private trackers. Also, there are trackers, that are manually added by the user..

[arvidn]

The specification doesn't mandate disconnecting any peers, at least not as far as I can tell.

The current use case for tiers, where they are all announced to independently, is have multiple trackers perform the same work, with the expectation that the majority of them will be down.

[ssiloti]

+1 on updating BEP 12 to match the de-facto standard behavior.

I think the current handling of events in libtorrent is probably the best choice. The other obvious option of sending one event per tier has much worse consequences if trackers within a tier do not share a backend.

In a ideal world tracker software would store a unique identifier on the backend and include that in every response. Then multiple trackers sharing a backend could be reliably identified.

[the8472]

Another aspect the specification is silent on is how &event=started and &event=stopped should be treated. Currently, libtorrent records whether it has sent a started event per tracker, regardless of tier. So this is not quite treating every tracker in a tier as just front-ends to the same database, since it will send started to the same tier multiple times, via different trackers.

I believe opentracker does data replication via multicast between multiple nodes that ought to be published within the same tier. So treating a single tier as load-balancing frontends would be correct in that case.

Any thoughts on what the behavior should be?

Well, ideally something that's not load-multiplying when users or torrent sites paste dozens of trackers into a client. There are some sites that just add more trackers to the announce list of existing torrents.

Maybe start with announcing to all tiers and then settling for the one with the largest client count or at least querying the others less frequently?

[the8472]

The specification doesn't mandate disconnecting any peers, at least not as far as I can tell.

But it does,

When a peer switches between trackers, the peer drops connections so that it cannot become an ongoing bridge between peers granted access from a private tracker and peers announcing to a public tracker. This partially mitigates the effect of an attacker modifying a metainfo file's announce-list and redistributing the metainfo file, e.g., via a public tracker web site.