holochain / holochain-proto

Holographic storage for distributed applications -- a validating monotonic DHT "backed" by authoritative hashchains for data provenance (a Ceptr sub-project)
http://holochain.org
GNU General Public License v3.0
878 stars 144 forks source link

Implement Cap'n Proto for wire format #222

Open artbrock opened 7 years ago

artbrock commented 7 years ago

Along with making all the fixed wire-types be Capn-proto specified, the dynamic Entry type needs taking into account:

Complete abstraction of Entry for different entry encoding types:

christopherreay commented 7 years ago

This branch: https://github.com/metacurrency/holochain/compare/222-implement-capn-proto-for-wire-format

I think before doing any more work integrating capnProto into the current code base, we need to do some hard research on the existing go-libp2p-dht-kad infrastructure.

Check these files:

https://github.com/libp2p/go-libp2p-kad-dht/tree/79b69919b8284b7bf06b9bd8812a7e1f1fd94777/pb https://github.com/libp2p/go-libp2p-kad-dht/blob/79b69919b8284b7bf06b9bd8812a7e1f1fd94777/dht_bootstrap.go https://github.com/libp2p/go-libp2p-kad-dht/blob/79b69919b8284b7bf06b9bd8812a7e1f1fd94777/handlers.go

Which seem to already integrate capnProto specifications into arbitrary message definitions.

Given the huge amount of existing code in go-libp2p-dht-kad around routing messages and maintaining the DHT integrity, I think, if possible, we should work on our own Neighbourhood Scheme (https://github.com/metacurrency/holochain/issues/263#issuecomment-314944715) on top of go-libp2p-dht-kad implementation.

I very much recommend we all put our heads together to analyse whether this is a plausible way forward sooner rather than later

@zippy @lucksus @artbrock

Also it would be nice if the branch creation and the commits in branch 222 were referenced into this issue :) xx

zippy commented 7 years ago

FWIW It looks to me like they (libp2p) went with protobuf, not capnproto.

Doesn't change that we need to figure out what we're doing about the DHT...

zippy commented 6 years ago

It's no longer clear that we will do this as we may be using demonsaw protocols instead...