ashwanthkumar / suuchi

सूचि - Toolkit to build Distributed Data Systems
https://ashwanthkumar.github.io/suuchi/
53 stars 12 forks source link

Optimize scans in VersionedStore #72

Closed ashwanthkumar closed 7 years ago

ashwanthkumar commented 7 years ago

Today as part of scanner and versionScanner we do a full store san and then filter for Data Keys or Version keys. This is very inefficient on very large stores and takes a long time. Instead we can always push the VERSION_KEY or DATA_KEY prefix to the underlying store there by reducing the search space of the whole scans.

ashwanthkumar commented 7 years ago

Closed via https://github.com/ashwanthkumar/suuchi/commit/5497c3fc56ef391d4e1727888fbef6dace0cca2b