search

ipfs / go-ipld-format

IPLD Node and Resolver interfaces in Go
https://github.com/ipld/ipld
MIT License
64 stars 26 forks source link

Suggestion for New IPLD Interface #32

Open patr-ck opened 6 years ago

patr-ck commented 6 years ago

Just some brainstorming here, but I believe the IPLD interface could be improved upon. I think it should be changed to the following ...

type Node interface {
  func String() string
  func Loggable() map[string]interface{}
  func Cid() cid.Cid
  func Links() []*cid.Cid
  func Tree(path string, depth int) []string
  func Resolve (path []string) (Node, []string, error)
  func Value() interface{}
}

type RootNode interface {
  Node

  func RawData() []byte
}

All previous "Node"s are essentially now "RootNode"s. Cid would need to be extended to include the path portion of block traversal (for example, a full Cid could be /ipfs/hash/path/inside/block). This allows each IPFS Block to actually represent a subtree of the overall merkledag, instead of just representing a single node + links and some unspecified "stuff" inside of its path. This way, all that "stuff" becomes Nodes in the merkledag.

The distinction of root nodes from other nodes separates the Block RawData method to prevent having to deserialize the block, and likely reserialize a subportion of it to get a []byte. Even HAVING a RawData representation of the subtree is sort of useless anyways, since hashing it to verify the serialization wouldn't even work (since it's actual serialization is somewhere inside it's RootNode's serialization), and creating a RootNode Cid of the form /ipfs/hash is also pretty useless since there may not even be a file on IPFS with that subtree's data serialized data.

The benefits of doing this are: it will allow standard IPFS dag tools to work with ALL SORTS of data structures that would otherwise have been painful to implement directly on IPFS. Any sort of tree like data structure could be stored directly on IPFS without the overhead of requiring an entirely new root node CID at every single link of the tree.

I believe this can be implemented as a generalization of the existing interface as well, since everything currently called a "Node" is essentially just a "RootNode" now, and (I believe) all existing Node types (git, bitcoin, ethereum, etc) SHOULD easily be adaptable to return Node's on Resolve instead of interface{}. This is because the Cid method is essentially free (it's just the Root CID + path), Strings and Loggable are easy, and Links, Tree and Resolve should be obvious as well.

Stebalien commented 6 years ago

See: https://github.com/ipld/specs/issues/3

I completely agree (although we would end up returning a Path from the children and only return CIDs from the roots, I don't want to start redefining what a CID is at this point). Unfortunately, fixing this at this point will be a bit hairy but we should still do it....

Stebalien commented 6 years ago

Note: I'd also have a method on interior nodes for getting the root.