search

bnclabs / gostore

Storage algorithms.
MIT License
35 stars 4 forks source link

BUBT: Cache all intermediate nodes. #47

Closed prataprc closed 6 years ago

prataprc commented 7 years ago

When Opening a well-formed bubt-index cache the intermediate nodes (m-blocks) in memory. Either using golang map or using llrb or some other fast lookup mechanism. It should typically be a map of fpos->mblock buffer.

Implement this as a configurable feature.

prataprc commented 6 years ago

With the latest re-arch of bubt, where intermediate nodes are stored in m-index file while leaf nodes are stored in z-index file, we can leverage on mmap feature supported by OpenSnapshot().

OpenSnapshot() will always open the m-index file in mmap mode. OpenSnapshot() will open z-index file and value-log (if present) in mmap mode only when requested by function argument.

mmap covers the caching requirement for the intermediate nodes.