Using MonoTorrent as a decentralized communication hub

[o2alexanderfedin]

@alanmcgovern I am looking for the following scenario:

1. A node (or a number of nodes) registers/announces a hash;
2. B node (or a number of nodes) looks up for peers that have registered/announced that hash;
3. B node(s) sends a custom request/message to the found peer(s);
4. A node(s) responds with a custom response/message to B node(s).

How can I do that via existing API?

[alanmcgovern]

While this is possible with a little bit of work, have you considered using something like RabbitMQ? MonoTorrent is a bittorrent library, and it's unlikely that it'll be a great fit as a decentralised communication hub.

From an implementation perspective, support for the libtorrent extension protocol is built in already, so the only thing lacking is an API which allows custom messages to be sent/received and also acted upon.

[o2alexanderfedin]

While this is possible with a little bit of work, have you considered using something like RabbitMQ? MonoTorrent is a bittorrent library, and it's unlikely that it'll be a great fit as a decentralised communication hub.

RabbitMQ is not decentralized. I am looking for a solution that can work even in completely isolated environment, and does not require any dedicated servers. Kademlia DHT serves the decentralization, reliability, and robustness very well. Having millions BitTorrent machines already running in the field makes it even more robust and reliable.

From an implementation perspective, support for the libtorrent extension protocol is built in already, so the only thing lacking is an API which allows custom messages to be sent/received and also acted upon.

My problem is that I don't know which strings (MonoTorrent APIs) to pull to get what I need to be done. If you could point me to the right APIs in the code base, that would be great. Can we do something like a Zoom call?

[alanmcgovern]

Having millions BitTorrent machines already running in the field makes it even more robust and reliable.

Maybe? BitTorrent DHT nodes store a mapping of infohash -> peer. While it's probably feasible to abuse that to store a list of fake infohash -> special peer, it's unlikely to be a great idea to piggy back non-torrent data on bittorrents dht - it's not designed for it.

That said - all you want for that is MonoTorrent.Dht. Create a DhtEngine, let it initialise, then call Announce or GetPeers as needed. That'll solve 1 and 2.

However 3 and 4 won't be solved with monotorrent. If you want to send non-bittorrent data to/from those peers then don't use a bittorrent library :p Perhaps you can build on some of the primitives by referning MonoTorrent.Messages, all of the code is fairly tailored towards bittorrent so it's hard to tell what'd be useful for a generic protocol.

[Arlodotexe]

IPFS seems a closer to what you're looking for than BitTorrent.

[o2alexanderfedin]

Maybe? BitTorrent DHT nodes store a mapping of infohash -> peer. While it's probably feasible to abuse that to store a list of fake infohash -> special peer, it's unlikely to be a great idea to piggy back non-torrent data on bittorrents dht - it's not designed for it.

IPFS uses similar approach, WebTorrent uses the same, so why not Torrent?

That said - all you want for that is MonoTorrent.Dht. Create a DhtEngine, let it initialise, then call Announce or GetPeers as needed. That'll solve 1 and 2.

If I create DhtEngine alone, it needs to be bootstrapped somehow, to get just a few initial connections to peers around. Also, I have to make the local ConnectionManager(?) listen to the incoming connections, which I also have no idea how to do. Will try to figure it out and probably get back to you with more questions.

However 3 and 4 won't be solved with monotorrent. If you want to send non-bittorrent data to/from those peers then don't use a bittorrent library :p Perhaps you can build on some of the primitives by referning MonoTorrent.Messages, all of the code is fairly tailored towards bittorrent so it's hard to tell what'd be useful for a generic protocol.

I can use one more layer of extensions to the ExtendedMessage, that is not a big deal.

[o2alexanderfedin]

IPFS seems a closer to what you're looking for than BitTorrent.

I am looking for a dotnet library or set of libraries that being actively developed, or at least supported. IPFS.net is dead, so it is not an option.

Also, there are millions of Torrent clients in the World, that can relay registration/discovery messages, and that makes this network quite reliable. I also have some plans to extend the lib with WebRTC connectivity support, similar to what WebTorrent has, and in this case that will be possible running the whole thing from within a web browser or Node.js as a Blazor WASM app.

[f5inet]

you need to write your own ExtendedMessage messages. For an IM app, you will need at least some persistent userID validation (prevent impersonation, also, bittorrent protocol randomize peerID at startup), so

• USERINFOASK (Asking user info, as ID, name, email and so on)
• USERINFORPY (reply message with the info asked in the previous message)
• CHATSEND (send chat message)
• CHATACK (acknowledge the received message)

This is the bare minimum. If you need groups, spam prevention, friends, encription by default... you'll need a lot more custom messages...

El mié, 31 jul 2024 a las 8:29, o2alexanderfedin @.***>) escribió:

IPFS seems a closer to what you're looking for than BitTorrent.

I am looking for a dotnet library or set of libraries that being actively developed, or at least supported. IPFS.net is dead, so it is not an option.

Also, there are millions of Torrent clients in the World, that can relay registration/discovery messages, and that makes this network quite reliable. I also have some plans to extend the lib with WebRTC connectivity support, similar to what WebTorrent has, and in this case that will be possible running the whole thing from within a web browser or Node.js as a Blazor WASM app.

— Reply to this email directly, view it on GitHub https://github.com/alanmcgovern/monotorrent/issues/662#issuecomment-2259768839, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMQLKR7W7XLN3G234EZQLDZPB74TAVCNFSM6AAAAABKYSOXS6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJZG43DQOBTHE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Arlodotexe]

I am looking for a dotnet library or set of libraries that being actively developed, or at least supported.

You're in luck. The best ipfs libraries on dotnet fell into disrepair after the maintainer passed away in 2019, but I brought them into the shipyard and have been working with the community the last few years to get things usable at a basic level.

• The api libraries in the Shipyard that are up to date and usable with the latest from protocol labs (Kubo, libp2p).
• To make it easier to download and bootstrap Kubo, see the OwlCore.Kubo library. There's other goodies in there, like pubsub encryption and implementations for the CommunityToolkit.Storage abstraction.

If you're trying to run in Wasm (Uno, Blazor, etc), you won't find many options. What these libraries cover so far is only enough to get Kubo working in dotnet, you might have to wrap around Helia if you're in the browser.

I'm slowly gathering people who can help build canonical implementations of multiformats, libp2p and ipfs in dotnet, but talent is fragmented and I've only just started. Hopefully in a few years, we won't have to bootstrap anything and can run on mobile and under wasm instead of just the [Windows, macOS, Linux] platforms offered by Kubo.

Still, ipfs was built to do exactly what you've described-- content addressing. They both use Kademlia-based DHTs, but ipfs does more (relay, bitswap, ipns, etc) and is made to be a suitable p2p alternative to http, more than torrents are. Worth a try!

[o2alexanderfedin]

I am looking for a dotnet library or set of libraries that being actively developed, or at least supported.

You're in luck. The best ipfs libraries on dotnet fell into disrepair after the maintainer passed away in 2019, but I brought them into the shipyard and have been working with the community the last few years to get things usable at a basic level.

• The api libraries in the Shipyard that are up to date and usable with the latest from protocol labs (Kubo, libp2p).
• To make it easier to download and bootstrap Kubo, see the OwlCore.Kubo library. There's other goodies in there, like pubsub encryption and implementations for the CommunityToolkit.Storage abstraction.

Just tried to run tests for OwlCore.Kubo on MacOS/X. There is no single one that succeeds. .NET 8.

Still, ipfs was built to do exactly what you've described-- content addressing. They both use Kademlia-based DHTs, but ipfs does more (relay, bitswap, ipns, etc) and is made to be a suitable p2p alternative to http, more than torrents are. Worth a try!

Tried. OwlCore.Kubo library does not work.

Actually, am I right that you never tried using PubSub API with this library? It reports that daemon is not started.

[Arlodotexe]

@o2alexanderfedin Glad to meet someone who's willing and able to test on MacOS. I don't own a MacOS machine, and while it should work I haven't been able to test there yet. Windows and Linux work, I've tested and made fixes for those platforms first-hand and plan to continue doing that if something breaks.

Regarding pubsub, I've used it and built additional tooling on top, like the aforementioned encryption layer. If you're seeing issues with the daemon not starting, it sounds like the issue is with the Kubo bootstrapper and not a specific API on Kubo.

If you'd like to help us fix the macos issue, we should continue the conversation in our community channel on discord since it's out of scope here: https://discord.gg/87sdNym8gC