Peer-to-peer applications

[purplesyringa]

In fact, this is not a bug but a feature request. So I probably won't follow the recommended issue format.

Idea

ZeroNet completely depends on sites. The only thing possible to be built on ZeroNet is a site.

However, I see a way to support much more. With the idea I promote we will be able to bring a blockchain here. The idea itself is rather simple: let's give the developer the control over P2P messages. For example, you could spread information to all peers, without files!

Implementation

Now to the implementation. There will be a new ZeroFrame command called peerBroadcast(message, peer_count=5, broadcast=True, immediate=False, timeout=60). This will:

1. Generate message ID by some_hash(str(rand()) + "," + json.dumps(message)).
2. Get peer_count random peers having current site.
3. Trigger peerBroadcast(message=message, hash=hash_from_p1, peer_count=peer_count, broadcast=broadcast, immediate=immediate) action on those peers.

P2P

When a peer receives peerBroadcast from another peer, it first verifies that current message wasn't yet received in current session (i.e. check it by hash in some dict or list). It's unlikely that the IDs will be duplicated (they may be, but it should be handled by the zite correctly). And it's unlikely that the message will be received twice after shutdown (i.e. get mesasge, shutdown, start again, connect to other peers, get message again). So we don't need to store the list in file system.

If the current message hasn't been received yet, it is sent to all current site's WebSockets via peerReceive(ip=from_ip, hash=message_hash, message=message) message. from_ip is IPv4, IPv6, or some other ID in case of Tor or I2P. So the site gets messages. The site may respond with another peerBroadcast message if it wants to.

immediate

If no WebSocket connection exists, e.g. the browser is closed, and immediate flag is set, the message is saved to the queue in memory. When the site is opened, the WebSocket connection is opened as well, and the queue is flushed to the socket. I say "in memory" because in fact if you shut down ZeroNet, other messages may be broadcasted, so you'll have to fetch them (e.g. by another peerBroadcast command).

broadcast

Finally, if the message hasn't been received yet (this prevents infinite loop), the message is broadcasted to some random peer_count peers. All the arguments are same as received.

Results

If broadcast is set to False, the count of peers will be rather small (peer_count or less if just a few peers are available). So we may get results from other peers (let's call them neighbours).

In this case, neighbours send the message to WebSockets, get the answer (via some another ZeroFrame command, e.g. peerReply), and return it to previous peer. The previous peer sets some timeout, i.e. in case that WebSocket isn't open or the remote side hangs. Finally, the gathered results are send in form of [{ip: ip1, reply: reply1}, ...]. reply is the exact reply from neighbour's WebSocket, and ip is either IPv4 or IPv6 or some other ID, e.g. in case of Tor/I2P.

Reply

Another ZeroFrame command called peerSend(ip, message) will send a message to peer specified by IP ip. The remote side may then reply, which results in return from actionPeerSend(self, to, ip, message), so the zite gets the reply.

Cross-site broadcast

It makes sense to ask user for permission to use CORS (because some site may store private data) and Merger (because it allows writing). However, just sending a message is secure enough. It makes no sense to do something serious when just a message is received. So as(..., "peer...", ...) should always be allowed for all sites, which will send a message to another site.

Spam spam spam spam

Sending messages to other peers may be unsafe. For example, peer A may send lots of messages to other peers. This leads to spamming their network traffic and wasting CPU power. So I'd like to add two levels of protection: passive and active.

Passive protection

A new entry called message_filter will be added to content.json. When a message is received from peer A, it is checked against message_filter regex in content.json. If the message doesn't match (i.e. it's invalid), it shouldn't be sent to WebSocket, to other peers. In this case peer A gets 5 ban points. So spamming won't be possible (no more than 20 invalid messages).

Active protection

The only kind of correct but spam messages is messages which are correct, but contain invalid data the remote side cannot handle. In case of Bitcoin this is invalid block hash. It's impossible to run any code in Python safely, so here comes active protection in browser. When a message is received from WebSocket, the zite may send peerInvalid(hash), which will add 10 ban points to the peer from which that message was received. This is more than passive protection because it's more likely to ban.

However, there is no way to reply to a broadcast message, so there is no way to check whether a message is correct without something like sleep; if didn't receive peerInvalid, then this message is valid. So instead we add peerValid(hash) command, which marks a message as valid, and will make broadcast a bit faster; however, if it isn't sent, it will be timeouted in say 5s, so messages will be transfered just a bit slower.

Usage examples

IRC

// Using my ZeroPage library, hopefully this will be easy to understand
zeroPage.on("peerReceive", ({params}) => {
    const {ip, message} = params;
    showNewMessage(message);
});

buttonSend.onclick = () => {
    zeroPage.cmd("peerBroadcast", {message: messageContent.value});
    showNewMessage(messageContent.value);
};

---

More examples are welcome.

---

Looking forward to hear from you, @HelloZeroNet.

[purplesyringa]

I'll probably make it a plugin.

[Thunder33345]

Your idea is solid but i think we should consider to be able to tie it to an ID if needed

Suggestion

To add optional native implementation to sign verify, encrypt, decrypt messages from/to an id

Possible Usage

Say i want to play tictactoe against another online user say bob@id.bit on a site, and we used P2P simply because it's not really needed for it to be permanent, we should have a native way to sign a broadcast message as current user, and check if it's a signed message, so we dont get our games messed up with other players that are also playing, maybe even encrypted to me@id.bit so no one can spy on how bad i am at playing tic tac toe

it's also possible for something like IRC-hybrid that only save the messages to disk once every 1min(or site close) to preserve history, but broadcast all messages P2P, since it's better to collect all writes and flush it all at once

Possible concerns

Concerns of privacy where one IP or tor address can be tied to one id, this can be dismissed, IF the site is big enough, more than likely that ip is just forwarding/propagating a message they receive

Minor Passive Protection Suggestion

Adding more passive protections, i dont think a regex is enough, we should have:

• something to restrict throughput

if i am sure my application will never send out more then 1 packet per second i can put it there to limit and stop people spamming, concerns: impossible to implement as anyone can just claim they are forward packets, and other peers will count your throughput to be your packets+packets you related but if there's a valid solution this will stop spammers dead in track maybe a way to check throughput against an sender ID if it's signed

• peer max byte to restrict max byte of an message

other peers will drop and flag/add points, if the message is over x byte(wrapper should reject sending if over this limit), this limit should be very high and is just there to to prevent users sending absurdly large packets and choking peers with less bandwidth

• force signed message

This flag force all messages sent to be signed with the valid signatures described(probably will be identical to the one in the data/users/content.json in most normal cases, unsigned messages will be dropped, this is useful for sites that know they wont be sending unsigned messages, thus making it harder to just spam with not signed packets without the whole peers starting to ignoring an ID, may not work for sign+encrypted messages unless you encrypt then sign so everyone know you send an encrypted message, rather then everyone just see an encrypted message but receiver can decrypt and see the signer

[purplesyringa]

As for passive protection, we should probably make it Etherium-like, i.e. allow crypto-functions, conditionals, etc.

[purplesyringa]

Renamed message_filter to p2p_filter.

[purplesyringa]

if i am sure my application will never send out more then 1 packet per second i can put it there to limit and stop people spamming,

Done.

[purplesyringa]

Added size limit. Will update docs soon.

[purplesyringa]

Implemented signature. (not encryption or "disable no-signature")

[Thunder33345]

do you think encryption via preshare keys are a good idea? (useful for things that need to get a group of people without having to re-encrypt to each recipient) not as secure as specified encryption

possible use case

I am playing a tic tac toe with bob@id, but i want to allow a few specific users to spectate, sure i could encrypt to viewer1@id and viewer1@id, but if the list gets absurdly large, it's just better to just have a public game in this case, but if i preshare the key from viewer1 to viewer100, it means keeping the game still private but allowing all viewers to view

It also works for a non bidirectional use case, say an invite only chat, where if the member grows everyone now have to do send one more encrypted message(so everyone can get it) unless everyone used a preshare key

Yes i used tictactoe in this example, but you can assume the same for more confidential things

sidenote, might be useful for partial encrypted leaving some metadata out, so everyone dont have to try to decrypt everything sent in their way say i want to not encrypt "gameid" so everyone dont have to bruteforce all the encrypted message received just to guess which one is my game, i can do

{
"gameid":"(immutable hash of something)",
"encrypted:"(encrypted json object)"
}

[purplesyringa]

Added p2p_signed_only.

[purplesyringa]

do you think encryption via preshare keys are a good idea?

It definitely is.

[purplesyringa]

@Thunder33345 It's still a question how encrypted messages should be decrypted when they are received - by P2P-messages plugin or by JavaScript?

[Thunder33345]

is an option in the root contents.json good idea? but i think try to decrypt fully encrypted by default unless declare otherwise tho need a way to tell the plugin of preshare keys to try to use, also would be nice to be able to be able to toggle passive or off for the decrypt while on js(need to retell each time the site is reopen)

parthial encrypted will be deal by js i assume since there's no way over it but there should be a decrypt utility class or something so user can pass the encrypted content into with possible key

[purplesyringa]

Maybe just use aesEncrypt/aesDecrypt instead?

[purplesyringa]

Problem: If the port is closed, no one can send you a message.

[Thunder33345]

hm so you need an open port to work? any workarounds that can be applied in an lower level/core itself to "fix" it?

[purplesyringa]

hm so you need an open port to work? any workarounds that can be applied in an lower level/core itself to "fix" it?

@krixano and I just checked it with only @krixano having port open, and it works. So if a peer has a closed port, but is connected to someone, this will work.

[Thunder33345]

Also we have problem of how to know if someone has a plugin using systemping will cause most nodes who dont have it to banpoint us we need a more general solution IMO