search

dotmesh-io / dotmesh

dotmesh (dm) is like git for your data volumes (databases, files etc) in Docker and Kubernetes
https://dotmesh.com
Apache License 2.0
539 stars 29 forks source link

Write up a design doc for not storing all our commit metadata in the KV store, RAM, returned in a single API call, etc... #694

Open alaric-dotmesh opened 5 years ago

alaric-dotmesh commented 5 years ago

As a dotmesh dev team, we'd like a path towards a world where we don't store all our commit metadata in the KV store, so that we can move towards being able to scale much better!

See https://discourse.dotscience.com/t/we-need-to-rethink-metadata-storage/34 for discussion on this topic.