Discussion around re-organizing the core protocol

[quartzjer]

I'd like to get a thread started around a new way to organize the core protocol concepts (with little or no breaking wire protocol changes).

At the highest level I'd like to propose reducing the core protocol to a smaller and lighter minimum of just the open/line/path mechanisms, and group all of the DHT functionality into it's own distinct area separate from it, such that implementations are clearly the core and DHT can be added/used by applications that need it. We need to clearly label/identify the core something simple like "TLP" (Telehash Line Protocol) and I'd like to combine the DHT with the local-area discovery/peering functionality into "TMesh" that together enable zero-dependency networking.

In order to make the core useful without the DHT I believe we need to introduce another role of a hashname that is a "router" (distinct from a "seed" that is still a role in the DHT) and clarify that a "route" is just which router can connect which hashname. This lets applications use the core with any architecture, either as a direct socket replacement or simple cloud-reflector setups. Adding a "route" is also the only thing that a mesh library needs to do as well and works as single the interface abstraction between them.

TLP would then consist of:

• no changes to the open, parts, keys, or line semantics
• only the "peer", "connect", and "path" channels
• clarification of "line" timeouts and keepalives
• normalize a tlp library api along the lines of makeSwitch(keys), .addHashname(parts,pubkey), .addPath(hn, path), .addRoute(router, hn), .channel(hn, type), .onChannel(...)
• multiple hashnames can be made default routers that are used when there's no specific routes
• creating a channel automatically does a connect via any routes if there's no paths active

TMesh would consist of:

• a mesh is created from a TLP switch
• bootstraps from seeds, uses buckets and seek/link channel (no change from current)
• is very dependent on the network transport (ipv4 is a different lib than bluetooth)
• broadcasting/listening is automatic
• informs app of local peers for discovery (and different level of trust)
• normalized api like makeMesh(switch), .seek(hn) returns routes, .onLocal(...)

Given our mission of making private communication channels, this feels like it makes that more accessible and easier to understand, lets apps adopt functionality as they need it, and makes it easier to implement each component.

Thoughts?

[dchote]

I like the sound of this.

[erickorb]

Me too!

"Trust only credentials that are TrueCred™ verified."

Eric Korb, President/CEO - accreditrust.com GoogleVoice: 908-248-4252 http://www.linkedin.com/in/erickorb

On Tue, Jun 10, 2014 at 6:45 AM, Daniel Chote notifications@github.com wrote:

I like the sound of this.

— Reply to this email directly or view it on GitHub https://github.com/telehash/telehash.org/issues/57#issuecomment-45598569 .

[amcjen]

I had to have a bit of discussion with Jeremie to get to the bottom of how this is different, but I get it now, and it makes a ton of sense--specifically with really low-power microcontrollers.

For those new to the Telehash world, it might be helpful to think of this proposed change as TLP being equivalent to something like TLS, and tmesh being equivalent to something like DHCP+Gateway. So you discover where you want to connect and how you're going to get there with tmesh, and TLP will make the actual socket connection between the two.

Two thumbs up on this one, looks like a great change.

[temas]

Having already had a decent discussion about this before the post, I too agree :)

[quartzjer]

Since I've been pretty quiet lately, I just wanted to kick out a little note to say that I've gone deep down this rabbit hole and have been regularly working over what a refactor like this could fully entail. I'm finally getting close to something that feels reasonable and stable enough to start discussing again :)

[dwaite]

I'm all for this; a smaller core protocol would allow the implementations (and the specs) to be much stronger. It also gives something for implementors like me to shoot for which doesn't require full DHT ;-)

If an implementation of just TLP still named a switch?

Is there a tier below TLP for an embedded system without a networking stack, or is this literally the core?

I assume reliable/stream channels will be part of TLP?

[quartzjer]

These are all just ideas I'm running with so far :)

Terminology is shifting a lot too with this refactor, there is no "switch" anymore, an implementation is just a library, and an instance is either a "router" or an "endpoint" depending on it's role.

The tier below I'm calling E3X and only implements the wire protocol and channel handling (including reliability logic), with a strict/limited API so that all network transport and application state is clearly separated.

[quartzjer]

I've started pulling together all of the discussion so far into a high-level summary, and am tentatively calling it V3 given the amount of refactoring. I've also started taking a detailed look/plan at the changes it would require in the JS and C implementations, and it's all feeling pretty good and straight forward, many simplifications.

For those that have been around telehash long enough, there's an interesting relationship between the versions:

• V1 - public only
• V2 - public and private blend
• V3 - private only

I'd love to get a sense from anyone if there's any major concerns jumping out at them before I start some branches and begin a more detailed reorganization of the protocol and implementations :)

[dchote]

I noticed that link is only for mesh establishment, but I feel that from a lib implementation standpoint there should be a base channel that can be used to determine connectivity and timeouts

[quartzjer]

Yep, that's how I expect the link channel to be used actually, it is the base channel for the lib to maintain connectivity, I just need to clarify it's updated definition a lot more in that it's not used for any "routing" information or interactions anymore and is specifically just for connectivity.

[dchote]

Gotcha. Yeah the new description of it had me thinking it was gone entirely for standard use

[quartzjer]

Let's move this discussion over here now for v3: https://github.com/telehash/telehash.org/pull/67