Use BitTorrent for file sharing

[yanmaani]

I think ZeroNet should transition over to using BitTorrent directly instead of trying to imitate it.

BitTorrent is a much more robust and well-developed protocol. I mean no disrespect to ZeroNet, but projects like libtorrent have very large budgets devoted to them, probably greater than all of ZeroNet, and it's just for transferring data and nothing else. BitTorrent can also handle larger files, as well as other things.

BitTorrent is an open standard. If ZeroNet started to use it, it'd be possible to interact with other BitTorrent-based projects. This is useful. For example, you could embed videos from BitTorrent directly into zites, and people could download zites in their BitTorrent clients.

If ZeroNet started using BitTorrent for new sites, it could focus on the core functionality, like building decentralized interactive websites, like using Tor instead of trying to re-invent the wheel. With that, making new implementations would also become much easier, because you could use a ready-made BitTorrent library instead of having to re-write the entire file sharing module.

Two semi-recent developments on BitTorrent have made this situation more interesting, and cleared away the last blockers:

• BEP 52 switches the hash algorithm from SHA-1 to SHA-256, changes hashing to the level of individual files, and implements something similar to TTH to get smaller torrents while keeping piece sizes at 16 KiB
• BEP 46 (or possibly BEP 50, or a custom solution as an extension) allows to dynamically update content

BEP 50 is a good example of this: ZeroNet handles updates (correct me if I'm wrong here) by naively gossiping them, whereas BEP 50 has a very clever structure for making sure updates are propagated in an efficient manner, basically creating an overlay network. Using BitTorrent allows ZeroNet to make use of all of these improvements, while still giving it the freedom to create extensions as needed. If useful, these extensions could be made into a BEP and pawned off onto upstream maintainers.

To be clear here, I'm not proposing that anything be explosively decommissioned, like Tor or Python v2. There's no urgent security flaw that mandates it.

[rllola]

I agree. Libtorrent V2 is making using BitTorrent in ZeroNet quite interesting.

ZeroNet Torrent plugin is a PoC toward that direction : https://github.com/rllola/zeronet-torrent-plugin

[yanmaani]

2063 relevant?

[wandrien]

BitTorrent doesn't play well with Tor. It will take some effort.

[yanmaani]

From my understanding, it should be fine. BitTorrent over Tor is bad because[1]:

1. It uses bandwidth
2. Current BitTorrent clients are badly implemented, because they simply ignore the proxy settings users have set:
   1. The default behavior was to "fail open".
   2. The proxy was sometimes not used for trackers.
   3. The proxy was sometimes not used for data traffic.
3. Without stream isolation, Tor can cause various leaks
4. BitTorrent prefers to use UDP instead of TCP, its TCP mode is inefficient, and Tor only supports TCP
5. Tor is at most efficient when data is sent in blocks evenly divisble by 512

In our case, this is...

1. True, but...
   1. So does current-day ZeroNet filesharing; BitTorrent doesn't use significantly more bandwidth, from what I know.
   2. Onion services do not need exit nodes, and that's where the capacity limit is, so using non-exit bandwidth should cause much less load.
2. Obviously not a problem, because we could use a library that did this properly and implement it in a non-broken fashion. This doesn't seem to be that difficult; libtorrent has flags for everything that people mention as a problem:
   1. This is no longer the case, but would have been possible to change by setting the force_proxy setting (now a no-op).
   2. This is fixed by enabling proxy_tracker_connections, which is the default now
   3. This is fixed by enabling proxy_peer_connections, which is the default now
3. libtorrent supports stream isolation by setting username and password for the proxy (type socks5_pw)
4. BitTorrent does support running over TCP, and blocking all UDP is trivial. It's probably less performant, but is it worse than the current system?
5. I don't think our current system is block-aligned, but maybe we should look into this.

In summary, I think it's good enough, but we should probably do some light validation that libtorrent does what it says on the tin. This shouldn't be hard.

[wandrien]

BTW, is there a python DHT library we can use for ZeroNet?

It should be properly isolated from the transport implementation details and should be compatible with gevent.

[yanmaani]

For what purpose if I may ask? Usually, DHTs are quite slow and unreliable, and there can be better solutions.

But if you have some reason, maybe you could try and combine asyncio-gevent and kademlia?