Draft: Wire protocol (DEP)

[pfrazee]

Rendered: Draft

Submitted for review. Working group will vote in next meeting (late February).

[bnewbold]

We should also write "Security and Privacy Concerns" and "References" sections for this one.

For references I could imagine:

• bittorrent, other "who are your influences?" protocol specs
• papers/references around choking and other scheduling issues (eg, see links from this ebay post
• more protobuf references (we have a single link right now)
• libsodium and hash/crypto algorithm references

For security/privacy:

• enumerate what a persistent observer can learn about peers
• enumerate some specific threat models and trust assumptions (eg, which properties of which hash and crypto algorithms are we assuming and relying on; distinction between, eg, targeted censorship and data manipulation/corruption)

[pfrazee]

Fixed, thanks

On Thu, May 31, 2018 at 3:26 AM Andrew Osheroff notifications@github.com wrote:

@andrewosh commented on this pull request.

In proposals/0000-wire-protocol.md https://github.com/datprotocol/DEPs/pull/8#discussion_r192023361:

+message Have {

• required uint64 start = 1;
• optional uint64 length = 2 [default = 1]; // defaults to 1
• optional bytes bitfield = 3; +} +```

• +#### Unhave +[msg-unhave]: #msg-unhave

• +type=4 Announces the loss of availability of Hypercore feed blocks for download. The start and length represent a continuous set of blocks.

• +This message is sent in the following contexts:

• • To inform the peer that data which was sent was rejected, for instance because it was not requested and pushes are not being allowed.
• • To announce blocks have been from local storage within a remote-wanted range.

"have been from local storage" -> "have been removed from local storage"

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/datprotocol/DEPs/pull/8#pullrequestreview-124712102, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNhUxNtWRR36JBgY7Zuvb0kR6LuVLiFks5t36k2gaJpZM4SbihG .

[bnewbold]

@pfrazee what's the status on this DEP? Are we blocked waiting on anything? If you move this somewhere I have git push privs I can try to push it through in the next couple weeks.

[pfrazee]

@bnewbold I think it just needs to get written, I don't think there are any blockers in dev work. I'll give you push rights on my fork.

[pfrazee]

Apologies for letting it stall!

[bnewbold]

Made some more progress on this, resolving most of the TODOs and comments. Still need clarification/feedback on:

• does ack flag actually do anything
• HMAC language

The security/privacy section in particular should get reviewed, and the whole thing probably needs a copy-edit and review for missing content (which I haven't done yet).

jIt would also be nice to have more examples, but those can take a long time to write up, and at this point, with effort being put in elsewhere (like the protocol book), I don't think it's necessary here for Draft status. There are no "rationale" or "alternatives" sections; I think those make more sense for new proposals and less for these older/existing protocol bits, but i'm open to objections.

Would be great to have this ready to vote, or at least review, by next WG meeting!

[pfrazee]

Updates look good to me!

[emilbayes]

I particularly liked the privacy/security discussion at the end. It was honest and covered all my "concerns". It's important to know that the public key is a very strong read capability, eg. that anyone with the public key can decrypt any traffic passively observed by any other peers communicating using that public key in their initial connection. So hypercores that are meant to be very sparsely replicated (eg. internet size ones) will leak a lot of metadata.

[bnewbold]

I think I've addressed everything above; we should give folks a few days to respond to things before a vote.

The block digest section should get review from an actual implementer (I never implemented that bit; cc: @mafintosh @yoshuawuyts).

Could probably also use another careful proof-read, but for Draft status I think it would be good to get this out and have eyes on it.

[bnewbold]

To set the timer, I will say I am submitting this for review to be voted on at the next WG meeting, though if there are significant changes needed we can push that off. I think we discussed "should we wait for any forthcoming implementation changes/refactors in the pipe" and my feeling is to get something out that documents current released/implemented behavior, and do an update or "v2" if needed.

[bnewbold]

Thanks for review anybody!

[urbien]

Was this Draft promoted to the Current (or however it is called) status?