search

webtorrent / webtorrent

⚡️ Streaming torrent client for the web
https://webtorrent.io
MIT License
29.65k stars 2.72k forks source link

WebTorrent DHT integration #1152

Open nazar-pc opened 7 years ago

nazar-pc commented 7 years ago

From discussion in #288 there is a potential implementation which I'd like to integrate with WebTorrent itself. The question now is how to do this?

We can more or less just replace bittorrent-dht/client with browser-compatible implementation, but desktop clients should likely participate in 2 DHTs at the same time.

My first idea was to create some kind of DHT proxy that will send requests and combine responses from several underlying DHTs, but I'm not sure this is a really good approach, since there are some features that do not translate directly this way (like dht.address() call).

Maintainers, what are your thoughts and vision about this?

RangerMauve commented 7 years ago

This is super cool! Is there anything in the way of getting this merged in? What are the next steps to getting this working everywhere?

nazar-pc commented 7 years ago

There are a few architectural decisions that need to be made by maintainers before this can be plugged in. You should be familiar with internals of related pieces in order to do something here.

Unfortunately my interest doesn't go much further that DHT itself (which is why I've made it) and I'm not familiar with WebTorrent architecture, so some help is definitely needed here.

RangerMauve commented 7 years ago

Is the DHT you've made functional on its own? Could one start using it to build a network?

nazar-pc commented 7 years ago

Exactly, just like BitTorrent DHT is functional on its own too. I've actually built it for another project and used WebTorrent's BitTorrent DHT implementation as the base for my implementation, which is why it shares most of the features with BitTorrent DHT.

I should note however, that there are some things to be done there, like dropping unnecessary WebRTC connection before browser tab crashes (both Firefox and Chrome simply crash with large number of connections), but it is ready for experiments with a few tens of nodes and can be improved later without breaking its API. Follow the repo for future updates.

Marak commented 7 years ago

@nazar-pc

It's very good to see someone working on this.

I believe that WebRTC based DHT is going to be a requirement for this p2p music sharing platform we've been working on ( https://github.com/lolashare/lolashare )

Looking forward to seeing if we can get more support for this feature. We may start testing out https://github.com/nazar-pc/webtorrent-dht soon.

Related: https://github.com/LolaShare/LolaShare/issues/45

RangerMauve commented 7 years ago

There's already support for the Kademlia DHT in node.js or electron main process based clients.

neuronsupport commented 6 years ago

It may be possible to get the DHT implementation from the IPFS-JS when it is done for web browsers. I am not sure how easy the integration of those bits can be.

DHT: https://github.com/ipfs/js-ipfs/pull/856 Relay: https://github.com/ipfs/js-ipfs/pull/1063

nazar-pc commented 6 years ago

How similar is it to Mainline DHT? If it is significantly different, it might make integration much more difficult, and even with almost identical implementation there is not much interest.

BTW, WebTorrent-DHT works pretty good now, though some improvements are still expected in node-webrtc (all reported upstream and the progress over last few months is amazing). There is a Detox project (transport package in particular) I'm working on that uses it under the hood, which might explain some more advanced uses of it (Detox uses different hash function, has cryptographic signatures on signaling data, multiplexes different data over the same data channel, etc).

I'm no longer interested in making integration into WebTorrent client myself, but I'd be happy to help if some feel adventurous enough.

neuronsupport commented 6 years ago

Does WebTorrent-DHT work inside the browser?

nazar-pc commented 6 years ago

Yes, it does. In fact, this is the reason it was created.

DiegoRBaquero commented 6 years ago

@norzak No, in the browser we use websocket-based tracker to share WebRTC offers and answers.

@nazar-pc ;)

neuronsupport commented 6 years ago

@nazar-pc I think WebTorrent-DHT is a separate project(in fact, I think your project) and is not part of the WebTorrent. I was referring to the WebTorrent project itself. I may have chosen the wrong issue to comment.

stale[bot] commented 6 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

bzz commented 3 years ago

Sorry for digging this up, but in case anyone else lands here from the search - did I get it right that ATM WebTorrent in the browser does not support peer discovery through DHT?

The documentation seems to suggest that it should, but the personal experience of trying to debug why a quickstart with the magnet link containing only the hash does nothing (and the fact that this issue is open) seems to suggest the opposite.

Update: after a bit more digging it seems that indeed DHT based discovery does not work in the browser (WebRTC DHT is discussed in #288) and in general, in-browser implementation only works with the ws://-based tracker.