search

chihaya / chihaya

A customizable, multi-protocol BitTorrent Tracker
https://chihaya.io
Other
1.43k stars 190 forks source link

Who is blame that other peers on a torrent do not reach me? #430

Open elico opened 5 years ago

elico commented 5 years ago

OK so since it's a tracker software it's a good place to list. I started looking at: https://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients But the list there is too much to grasp. I know about:

I have also recently seen:

But everyone of these have something "special" about their implementation. One connects to a specific HUB while the other use specific "protocol".

The other weird thing was with these is how the load is distributed between the peers. If I take a torrent of let say Ubuntu-1604 server iso file and start downloading and hosting it for some reason after I have finished downloading the torrent and there are still peers which didn't finished the download, not one of them contact's my torrent software.

I am not sure what is causing this effect and was wondering if a tracker have any fault in it?

mrd0ll4r commented 5 years ago

Alright, let's see.

General distinction between clients:

  1. Clients that use the standard bittorrent wire protocol as described in BEP 3. Clients of this family are transmission, libtorrent, rtorrent (which uses libtorrent I think), etc.
  2. Clients that use the webtorrent protocol. This is very similar to the standard protocol, but uses webRTC as transport.
  3. Possible hybrid clients (I think Vuze was/is able to do both by using a headless chrome for the webtorrent stuff, but don't quote me on that).

Bittorrent and webtorrent clients can not talk to each other. They need a hybrid or bridge client that supports both protocols.

Webtorrent clients need a special tracker that

  1. Supports communication with webtorrent clients (probably webrtc, maybe something else)
  2. Supports the slightly changed tracker protocol (which is undocumented)
  3. Distinguishes webtorrent and non-webtorrent clients, as they should not be mixed (because they cannot talk to each other)

See https://camo.githubusercontent.com/3f57ad9ba26fad961f316c2eb20d5b9f12341b6e/68747470733a2f2f776562746f7272656e742e696f2f696d672f6e6574776f726b2e706e67.

Generally, reasons for why you're not seeding could be:

  1. You're not connectable (port blocked somewhere along the line, NAT, ...)
  2. The download strategy/choking/unchoking algorithms decided not to download from you
  3. The client is connected to enough other peers

Generally, when you're seeding. you still send announces to the tracker and receive peers. If the tracker optimizes (chihaya does), you'll receive only leechers (=incomplete peers) in the response. Your client will display this list of peers, but ultimately it's usually up to leechers to connect to you to download, not the other way around. I'm not 100% sure on thlis though, but I'm pretty sure you cannot push data to them (meaning you might be able to connect to them, which would for example tell you which client they're running, but you cannot force them to download from you.)

Hope that helps a bit :)

elico commented 5 years ago

@mrd0ll4r helps indeed. If I was able to push data to clients it was a series issue. I kind of knew the basics but it seems like the concept of P2P and torrent specifically was drastically changed since 1995-2k.

One thing I have seen is something like Tracker-less torrent system but I believe it's bogus. I am pretty sure there are ways to solicit a peer to download from you.

I liked the optimization code..

mrd0ll4r commented 5 years ago

One thing I have seen is something like Tracker-less torrent system but I believe it's bogus.

Oh no that's totally a thing! You get the list of peers from a DHT, then just connect to them and proceed as usual. We even have torrent-less torrents now (without the torrent file), clients download that from a peer they found on the DHT, see BEP 9 :)

Trackers are only mandatory for private trackers and to make things faster now. Or for people who disable the DHT.

elico commented 5 years ago

@mrd0ll4r So let say I want to "host" only the metadata files without the torrents content files on a DHT node I only need to have a bunch of torrent files and a specific torrent server/client which is well connected to a or the DHT network.

Now... to the other subject I have seen on the link:

v1: magnet:?xt=urn:btih:<info-hash>&dn=<name>&tr=<tracker-url>&x.pe=<peer-address>
v2: magnet:?xt=urn:btmh:<tagged-info-hash>&dn=<name>&tr=<tracker-url>&x.pe=<peer-address>

I have seen mainly the v1 format but I am struggling to understand how would a tagged-info-hash tagged-info-hash would look like and how it would be generated. Also what client/server software have implemented v2?

mrd0ll4r commented 5 years ago

I have honestly never seen a v2 magnet link, what's the specification for the btmh field?

f4nff commented 5 years ago

@mrd0ll4r help #432

elico commented 5 years ago

@mrd0ll4r I really do not remember. it should support multiple torrent file hashes ie sha1, sha256, sha512 or any other that is specified and can contain multiple of these.

polarathene commented 3 years ago

v2 torrents are still not as widely supported AFAIK. libtorrent 2.x has support for such since 2020 IIRC, while some clients like qBitTorrent that use libtorrent under the hood are presently working on integrating support for that on their end.

There is also hybrid torrents which have v1 and v2, they're intended to provide fallback compatibility for broader client support, but some clients like Transmission are known to break on that. v2 SHA-256 hashes are truncated to be the same size as SHA-1 hashes from v1 iirc (might just be the infohash), it's detailed in the libtorrent blogpost from 2020 about it (which details other changes/features).

WebTorrent clients (without hybrid support to bridge regular non-WebTorrent clients) use WebSockets to connect to a tracker (wss://) and then WebRTC for connecting to peers and transferring data. Some things to be aware of is extensions (BEP?):

A common issue with magnet links can be a lack of metadata needed that a .torrent file provides. WebTorrent content which only has webtorrent tracker can only serve webtorrent clients (who generally can't use trackerless alternatives via DHT and PEX), whilst some magnet links provide params for xs and ws to help kickstart a transfer without waiting on peer discovery by downloading from a webseed (HTTP web server) directly (and later connecting to any available peers in a swarm).

Quite a few desktop traditional torrent clients AFAIK don't parse and handle these params properly (but they may support webseed via .torrent), otherwise even without WebTorrent tracker and peers, you would be able to retrieve the content from the webseed AFAIK.

Likewise, a webtorrent only client would have trouble connecting to non-webtorrent capable clients and trackers, and thus have difficulty interacting with peers as well.

For reasons stated above, you can therefore find a torrent that doesn't seem to work and no leeching, or you could leech from a webseed and find that you can't seed to anyone.

Could be due to other reasons, but these are issues I've come across reading various Github issues and scenarios.