janelia-flyem / dvid

Distributed, Versioned, Image-oriented Dataservice
http://dvid.io
Other
196 stars 33 forks source link

Metadata for values in a keyvalue instance #294

Open stuarteberg opened 5 years ago

stuarteberg commented 5 years ago

For certain use-cases, we will need some way of storing metadata along with values that we store in keyvalue instances.

(As a stop-gap solution in the past, we have occasionally stored a separate '_info' key alongside every value that needs metadata, but that's non-optimal for multiple reasons.)

Ideally, the metadata for each value would be:

  1. Optional.
  2. Easily posted simultaneously with the value.
  3. Stored contiguously with the value within the database.
  4. Not returned by default for an ordinary .../key/... call.

If we choose to implement this feature using protobuf, then a protobuf-based read/write API could be exposed for powerusers. But either way, a "simpler" API based on pure HTTP will be needed for naive clients who won't want to get their hands dirty with protobuf. Possible "simple" implementations could include: