greglook / blocks

Clojure content-addressable data storage.
The Unlicense
113 stars 6 forks source link

Block enumerator protocol #8

Closed greglook closed 8 years ago

greglook commented 9 years ago

Per this TODO comment:

Protocol which returns a lazy sequence of every block in the store, along with an opaque marker which can be used to resume the stream in the same position. Blocks are explicitly not returned in any defined order; it is assumed the store will enumerate them in the most efficient order available.

This is mostly useful for synchronizing two block stores, as the data will be read out of the source in an efficient order. For example, if using a diskpacked-like store you don't want to have to skip between packfiles frequently just to access the blocks in ascending identifier order.