DEP: Session data extension

[pfrazee]

New DEP proposal. This hasn't been discussed in the WG yet, but came from discussions in #dat on freenode about discovery. The summary:

This DEP defines the non-standard session-data extension message used in the Dat replication protocol. This message provides a way to attach application data to a connection, commonly used for identifying the users and broadcasting personal keys.

The use-case is similar to what @cabal-club has done with the handshake's peerId and userData. I explain in the DEP why I diverged from that approach, and I hope that cabal will be able to adopt this as well so that cabal can work in Beaker.

cc Cabal team @karissa @cblgh @noffle as well as the dat WG as a whole

[mafintosh]

Looks very clear to me. LGTM

[mafintosh]

Or ... LGTV (looks good to vote)

[hackergrrl]

Hey y'all. :wave:

The reason we use this on-connection hack in cabal is so that we can auto-authorize and get around these hyperdb issues (131) (132). The proposed change here would help cabal work in Beaker, though removing the hyperdb requirement of having a pre-established shared key and explicit authorization would obviate the need for such a DEP (for cabal or any other app that doesn't want a mandatory authorization model, at least).

[RangerMauve]

Yeah, an auto-authorized hyperdb could act as a communication channel if people listen to the changefeed.

[okdistribute]

Yeah, that's correct @noffle

[pfrazee]

Hm, I'm concerned about auto-authorizing hyperdbs, for scaling and (perhaps more importantly) for spam attacks. Like: What happens if somebody shows up and starts dumping large datasets into it? Is there a way to stop that and remove the data they add?

I like how elegant the idea is though. It'd be great if every app could have a public hyperdb to coordinate and discover. I'm just not sure we can make it work.

What I was trying to enable with this DEP is two things:

1. For small apps to be able to discover peers when shared. Things like, a dat containing an event invite, or a collaborative document. The idea being that scale would be kept small by the fact that the dat is only shared with friends. If too many people start showing up, the app could stop authorizing or downloading their data.
2. For apps like cabal and fritter to experiment with more risky policies, like auto-adding.

It's also important to note -- this DEP was motivated by the need for something that works in the near-term. I expect it to be a stepping stone to a more sophisticated solution.

[hackergrrl]

I like the idea of the dat/DEP core pieces not worrying too much about policy matters, like authorization, which many different userland app will have different opinions on and needs from. I wonder if it'd make sense to hold off on this DEP until hyperdb makes user-implemented authorization models possible, and see if this DEP is still necessary?

imho specs are so much heavier than a module doing a semver bump to add/remove an API; my preference is to explore the latter option first.

[pfrazee]

I’m 👍 to keep discussing multiwriter’s auth policies but I have utility for this spec outside of multiwriter authorization so I’d prefer we didn’t hold up this DEP for that discussion.

[RangerMauve]

@pfrazee What sort of API were you aiming for from a beaker perspective? It doesn't look like this would be used as a PeerSocket.

[RangerMauve]

If it'll be using the same-origin policy, what will that mean for DNS?

If I publish session data on dat://fritter.hashbase.io, will that conflict with dat://9900f9aad4d6e79e0beb1c46333852b99829e4dfcdfa9b690eeeab3c367c1b9a?

[pfrazee]

@RangerMauve I'm still thinking about the API, but I'll write something up soon. I'm also going to write up a DEP for ephemeral messages that's similar to this, and the overall API design will depend on that too.

The domain names will not have an effect in this case. Beaker will resolve the current site to its raw URL, and only allow that site to access the session-data for itself.

[bnewbold]

I missed the boat on reviewing this earlier, but a few notes anyways:

"Any additional bytes should be truncated by the receiving client": I wouldn't ever truncate messages; this leads to debugging nightmares (see also packet fragmentation and UDP MTU truncation). I'd recommend dropping the whole connection ("fail fast"), or disregarding the entire message (though even the later could also be hard to debug, especially if session messages have variable size). None of these are actually great solutions though... what is the upgrade mechanism to expanding the size or complexity of this field in the future, while being backwards compatible with older clients/agents?

I guess overall this feels under-motivated to me, so it's hard to judge whether it delivers on the goal it sets out to achieve. This is probably because I am focused on Dat as a protocol for replicating published content (where I don't think this DEP would have much to provide) and haven't thought as much about real-time and ephemeral use cases as much, so I can't be as helpful.

[pfrazee]

None of these are actually great solutions though

It's definitely a concern that a truncated message could be misinterpreted. It might be a better idea to drop the message and suggest the receiving client emit an error event, to potentially react to.

what is the upgrade mechanism to expanding the size or complexity of this field in the future, while being backwards compatible with older clients/agents?

Another extension message, like session-data-v2

I guess overall this feels under-motivated to me, so it's hard to judge whether it delivers on the goal it sets out to achieve.

Can you be more specific about what you want? This is kind of a frustratingly broad criticism.

[bnewbold]

I guess overall this feels under-motivated to me, so it's hard to judge whether it delivers on the goal it sets out to achieve.

Can you be more specific about what you want? This is kind of a frustratingly broad criticism.

That's totally fair! I think this is a case of not communicating context well over the internet. I think i'm looking for more context around "where does this fit in the big picture". Is this DEP a way to document and formalize what a couple apps are already doing? Seems reasonable. Is it setting out The default way to authenticate hypercore peers as identified users? That seems ambitious and i'd want to do more reading.

Edit: I also commented after only reading the DEP itself. Your comment https://github.com/datprotocol/DEPs/pull/27#issuecomment-394051467 above and https://github.com/datprotocol/DEPs/pull/27#issuecomment-393978297 provide a lot more context.

[RangerMauve]

Personally, I'm going to be using this for discovering peers in an application.

For example, it can be used in a social media setting to discover peer's day URLs to automatically index them.

[pfrazee]

@bnewbold okay that makes sense, does https://github.com/datprotocol/DEPs/pull/30 improve that you think?