Generic `ipfs p2p send` API

[d180cf]

I think IPFS would benefit from a generic messaging API:

• ipfs p2p send <node-id> <message-data>
• ipfs p2p block <node-id>
• ipfs p2p unblock <node-id>

There should be a way to listen to incoming messages, allow or block particular senders and so on.

Perhaps we can extend the definition of ipfs pubsub and say that ipfs pubsub pub <node-id> <message-data> is same as sending <message-data> to <node-id> directly.

Use case: p2p apps that use IPFS.

Example: a reddit-like app to publish, read and comment news. Content is stored in IPFS, but new comments and posts are shared via the p2p messages:

• Someone wants to make a comment under post 1e662bcf1
• The comment is sent to a few peers.
• The peers verify that the comment is ok and re-transmit it further.

Why can't we just use ipfs pubsub? It's impossible to stop flood in the pubsub model without introducing some sort of authorization mechanism, which is not flexible enough for all use cases. For example we want to allow anyone to post comments as long as they are accepted by a script running on each ipfs node participating in this reddit-like system. In this model there is no central authority granting or denying the permission to post comments based on node identity.

However ipfs pubsub is useful and necessary in this system. For example, when a new node joins the system, it needs to find other nodes that already participate. It starts broadcasting messages to 1e662bcf1 saying

• "respond directly if your ipfs id differs from mine by 1 bit only"
• "respond directly if your ipfs id differs from mine by 2 bits"
• "respond directly if your ipfs id differs from mine by 3 bits"
• and so on, until 100 or so peers are found

Responding directly means ipfs p2p send. There can be millions of participants and it won't be nice if they all do ipfs pubsub pub.

You might ask why don't we just use ipfs pubsub the same way to publish new comments? Verifying new comments may be expensive: need to download the comment, run a verification script, possibly do other things. However if new comments are re-transmitted to other peers only after verification, then the potential DoS damage is limited to peers of the offending node, which will be blocked quickly.

Once a participant of 1e662bcf1 is found, it's time to sync the state. This can be done with ipfs p2p send:

• One option is to just put the list of comment hashes into a IPFS file and send the hash of that file to the other peer. Easy, robust, slow. Won't work if we want to sync lists with millions of items.
• Another option is to split the comments into two sets: B0 and B1, where B1 are those comments whose IPFS hash starts with 1, and B0 - the rest. Send hashes of the two sets to the participant: ipfs p2p send <participant-id> H(B1),H(B0). The participant does the same, compares the two hashes and sees which sets differ. If they differ, the process continues recursively.

That's the outline of one possible idea. I'm sure there are many other p2p apps that will need this.

[magik6k]

AFAIK pubsub should be able to support your use case with a few extensions, though I'm not exactly sure how hard would that be. ccing @vyzo for more info

I agree that implementing p2p send would be useful, but that's something that will probably need to wait for messaging layer in libp2p.

[vyzo]

this is awfully close to simply using pubsub as it exists right now.

[Stebalien]

• Pubsub is getting signature support.
• You can already build this with the ipfs p2p commands.

However, I agree that a simpler, message oriented ipfs p2p command would be interesting. One could call ipfs p2p message listen /some/protocol/1.0.0 (or something like that) to receive a stream of incoming messages in the form {from: QmId, data: data}.

[d180cf]

I didn't know about ipfs p2p. That's pretty cool. As long as it allows to get messages from arbitrary ipfs nodes without establishing some sort of channel or connection beforehand, it will be enough, indeed.

[rklaehn]

I would prefer the existing functionality to be polished, made consistent over the different implementations, and heavily performance optimised before adding new features. IPFS, IPNS and pubsub already allow you to build some very sophisticated distributed apps.

But currently pubsub is experimental, IPNS is slow and frequently broken, and the distributed content addressed storage abstraction is far from as fast as it could be. So just IMHO the desire to add more and more features before making the existing features solid is very misguided.

[Stebalien]

So just IMHO the desire to add more and more features before making the existing features solid is very misguided.

This 100%. However, features like this can be implemented without too much effort by new contributors just trying to learn the codebase so we shouldn't block new features on polishing existing ones.

[Stebalien]

Well, shouldn't take too much effort. Seemingly simple features can sometimes turn out to be really complex.