Collaborate with other p2p projects

[Raynos]

• topology

I'm trying to work on arbitary p2p topology construction code. To do that a topology needs a list of possible peers in the network and a mechanism for connecting to peers.

If we can standardize on an API for these building blocks we can build re-usable code!

[simonlast]

That sounds fun! I've been playing around with this library for a bit; I'll see what I can come up with.

[piranna]

I totally agree, I have been modifying lately my ShareIt! project just in that direction

[michelle]

We should talk about how this standard API should look.

[piranna]

I'm currently writting the specification files of my ShareIt! project, and also the one related to the most internal layer that's the one that I was thinking would be "merged" with PeerJS code, only that I'm focusing it on use third-party channels as handshake servers (currently I'm using PubNub, I hope to use in the future anonimous XMPP servers) and using a "W3C-like" API instead of a socket.io one as PeerJS does. You can find more info on the discussion I started on the WebP2P mail list (http://librelist.com/browser/webp2p, although it seems they are not being showed the last ones... :-/ ).

[Raynos]

I'll be at the webrtc meetup in SF tomorrow. We can talk about the problem scope then.

[piranna]

Sorry, I'm at Spain and can't move from here (money & university classes :-( ).

[autarc]

Can't join either, but count me in for further planning. Even though gaming is a bit more specific, considering the different approaches for a lobby & in-game. I would like to help defining a baseline for general handling as well, easing the setup and customization.

[piranna]

I have just finished to write the specification of ShareIt! inner protocol for handshake and routing (what I call "WebP2P"), and also the other day I did the same for the transfer protocol. You can find them at

https://github.com/piranna/ShareIt/blob/master/js/webp2p/doc/draft-piranna-webp2p-00.txt https://github.com/piranna/ShareIt/blob/master/js/shareit-core/doc/draft-piranna-webp2p_ftp-00.txt

After writting the specification I have find they are really far from perfect, but it's a first step :-) In any case, somebody told me some time ago that ShareIt! architecture was similar to Gnutella project and I have been looking for a copy of the specification of Gnutella protocol:

www.stanford.edu/class/cs244b/gnutella_protocol_0.4.pdf

It's not the latest one (the latest one seems to be http://rfc-gnutella.sourceforge.net/src/rfc-0_6-draft.html, but also seems it was never finished) but up to some point it would be used as basis to develop a DataChannels-based P2P protocol. It also split from the discovering and connection from the filesharing layers, and ironically, for filesharing they use HTTP :-P

Please give me your comments.

[Raynos]

@piranna offer & answer is not enough. You also want to send candidates.

As for your presence tactic, I don't know what's ideal for discovering network topology. I feel like you would want meta data about the nodes in the network before you start making connetions.

Which is why I went with the idea of a replicated list of nodes and their meta data ( https://github.com/Raynos/peer-nodes#example ) held on a central server. This allows for efficient creation of network topologies at the cost of having to manage the list.

Really I want two things. I want to be able to open a stream and assemble a life data representation of the real network topology and have some minimal machinery for sending offers/answers/candidates.

It's specifying some kind of protocol for the former that's difficult

[piranna]

@piranna https://github.com/piranna offer & answer is not enough. You also want to send candidates.

I wasn't thinked about candidates because I didn't required them, but seems they will be needed finally for DataChannels too (I hope don't...). Anyway, they would be required for audio & video. Do you know more about how to use them? Could you be able to give me some advice or fix it yourself? :-)

As for your presence tactic, I don't know what's ideal for discovering

network topology. I feel like you would want meta data about the nodes in the network before you start making connetions.

I don't think so. I'm not a networks expert, but my idea was to make a random mesh, starting getting some connections from the handshake server and later leave it and start working just over the P2P network, making direct connections as it's required. If you want some kind of concept, think of it like peers are entering on a queue (the handshake server), there the get connections from older and newer peers, and when later they got out of the queue they start to create the real random mesh according to the peers to whom they get connected.

Which is why I went with the idea of a replicated list of nodes and their

meta data ( https://github.com/Raynos/peer-nodes#example ) held on a central server. This allows for efficient creation of network topologies at the cost of having to manage the list.

Central server? No way, at least for me. A central server would be a big single-point-of-failure, and this would be a big failure on the design. We should go fully decentralized so it would have better opportunities to survive. Also, having a central server (and just thinking about it) would be a difficult issue for peers anonimity. I would like it to be fully serverless and I have been developing ShareIt! this way (the only "central" point is the current PubNub handshake server, and I'm thinking about testing with anonimous XMPP servers to deprecate it), but if isn't possible, what about a federated system?

Really I want two things. I want to be able to open a stream and assemble a

life data representation of the real network topology and have some minimal machinery for sending offers/answers/candidates.

It's specifying some kind of protocol for the former that's difficult

I agree about the point of have a simple API that can do the offers/answers/candidates easier to do, and ideally it should be transparent (at least for me), but the idea about has a life representation of the network topology, not only it's difficult but I also find it useless or maybe also dangerous for peers anonimity, that I believe it should be one of the main purposes of the P2P network. Maybe we have conflicting use cases here?

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton de sitios diferentes, simplemente escribe un sistema operativo Unix." – Linus Tordvals, creador del sistema operativo Linux

[Raynos]

@piranna setting up a random mesh like that can work but it's expensive. It depends what kind of application you are building.

For file sharing & torrents I don't mind that the first 5 minutes are useless because its travelling the network finding an optimum topology for downloading the 4GB file.

For a real time application I want to be able to connect and have my state saturated from relevant peers within seconds. Using a known distributed, de-centralized system that's in charge of maintaining topology information reduces my application start up time.

[piranna]

Interesting point... maybe in that case a central server (or federated servers, or cluster) would be a better aproach. Maybe we can have both. At a first point, I think it would be feasable having a pure decentralized network like the one I propose, and later have the option for some specific applications like your to have servers that manages the connections. Also, they would be used to improve the initial handshake and bootstraping and later disconnect in a use case like mine, too. The problem here is that we come back to the single-point-of-failure, but being an optional component to assist and improve management and not a central one on the architecture desing, it could work. What do you think?

2013/3/12 Raynos notifications@github.com

@piranna https://github.com/piranna setting up a random mesh like that can work but it's expensive. It depends what kind of application you are building.

For file sharing & torrents I don't mind that the first 5 minutes are useless because its travelling the network finding an optimum topology for downloading the 4GB file.

For a real time application I want to be able to connect and have my state saturated from relevant peers within seconds. Using a known distributed, de-centralized system that's in charge of maintaining topology information reduces my application start up time.

— Reply to this email directly or view it on GitHubhttps://github.com/peers/peerjs/issues/7#issuecomment-14752915 .

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton de sitios diferentes, simplemente escribe un sistema operativo Unix." – Linus Tordvals, creador del sistema operativo Linux

[Raynos]

@piranna we still have a single point of failure, the handshake server.

To mitigate that you should be able to use ANY handshake server or multiple handshake servers and we shouldn't require for every peer to use the same handshake server.

Which means you need to be able to ask any peer you have an OPEN connection to (after you disconnected from the handshake server) to carry your offer/answer/candidates and to carry your presence information.

i.e. whatever communication you do over the handshake (BOOTSTRAPPING) server must be able to be done in a pure p2p fashion.

[Raynos]

@piranna we may converge to creating a DHT over RTCPeerConnection that stores topology / presence information or at least allows for efficient propagation of messages that allow for creating a non-weighted random topology

[piranna]

@piranna we still have a single point of failure, the handshake server.

To mitigate that you should be able to use ANY handshake server or multiple handshake servers and we shouldn't require for every peer to use the same handshake server.

I have this in mind since several months ago :-) In fact, the HandshakeManager is ready to use several handshake servers at a time, only that currently I'm using only one (PubNub) because I didn't yet got how to federate different ones between themselves. Also I didn't be able to work on it :-P but I was able to use PubNub or SimpleSignaling just changing a JSON file, it only keeped to find a way to use any of them. One solution would be to open connections to several handshake servers and listen to them for new peers, but only notify presence on one of them. This way it would be server-down secure, but also it would be somewhat easy to detect a high activity and block it. My idea is to connect just to one of them sequentially, but this would require to have a lot of peers on the network so you don't get connected to an empty handshake server... :-/

Which means you need to be able to ask any peer you have an OPEN connection to (after you disconnected from the handshake server) to carry your offer/answer/candidates and to carry your presence information.

i.e. whatever communication you do over the handshake (BOOTSTRAPPING) server must be able to be done in a pure p2p fashion.

This thing is working, too. Routing.js is the file that manages this, and it's applied both to handshake server channels and DataChannels :-)

[piranna]

@piranna we may converge to creating a DHT over RTCPeerConnection that stores topology / presence information or at least allows for efficient propagation of messages that allow for creating a non-weighted random topology

Originally my idea was to use KadOH (Kademila Over Http) but developments seems got starved... :-/

[IvnMO]

Check : http://www.nodetron.com/

we are going to help