search

ipfs / kubo

An IPFS implementation in Go
https://docs.ipfs.tech/how-to/command-line-quick-start/
Other
16.17k stars 3.02k forks source link

Can I use IPFS DHT to store arbitrary key-value pairs? #2387

Open sivachandran opened 8 years ago

sivachandran commented 8 years ago

I basically want to store key-value pairs in the distributed network and later retrieve value for a specific key. A related topic is already discussed here https://github.com/ipfs/notes/issues/4 but I am not sure about the current implementation status of it.

I understand IPFS stores the hash to object/block mapping in DHT using this package https://github.com/ipfs/go-ipfs/blob/master/routing/dht/dht.go. So I am wondering whether I can use the same DHT package to store my own key-value pairs. But my keys won't be hash of the value that I store in the DHT. As this is contrary to how IPFS uses the DHT I am not sure whether I can do this or will it create any problem in how IPFS functions.

So can someone tell me whether I can use the IPFS DHT to store arbitrary key-value pairs or is there a better way to do.

ianopolous commented 8 years ago

You could use a content-addressed btree. I implemented one at: https://github.com/ianopolous/merkle-btree Or there's a version specifically adapted to IPFS (using the Java HTTP API) at: https://github.com/ianopolous/Peergos/tree/master/src/peergos/server/merklebtree

Then you just need to pin the root of the btree.

hackergrrl commented 8 years ago

Here are some summarized items from @jbenet when I chatted with him recently about arbitrary kvpairs on the IPFS DHT:

  1. The point of records is that they will be validated in the future: you can store arbitrary value within a record. (eg signed or with some other validity function)
  2. This is critical to avoid DHT spam. We're in a safe DHT right now (read: not under attack).

In comparison to Mainline (bittorrent), which many nodes will let you know arbitrary data on:

  1. Mainline DHT lets you store whatever, but it both a) restricts the size of that junk, and also b) many nodes are very aggressive about dumping records.
  2. Mainline isn't attacked as hard as we think, because people actively monitor it instead, to find bad content and fine people. (copyright enforcement inspects).
  3. Also, Mainline is big. 15-30 million nodes big.