search

windelbouwman / lognplot

Plotting and logging of real-time data for desktop.
GNU General Public License v3.0
78 stars 11 forks source link

Storage backend using RocksDB database #14

Open TimonTerBraak opened 4 years ago

TimonTerBraak commented 4 years ago

The RocksDB library provides a persistent key value store. Keys and values are arbitrary byte arrays. The keys are ordered within the key value store according to a user-specified comparator function.

Implementation is in C++, there are rust and python binding available. The database is stored in a single directory (not single-file).

https://github.com/facebook/rocksdb/blob/gh-pages-old/intro.pdf?raw=true https://github.com/facebook/rocksdb/wiki

Python bindings for rocksdb: https://python-rocksdb.readthedocs.io/en/latest/index.html

Rust bindings for rocksdb: https://github.com/rust-rocksdb/rust-rocksdb

windelbouwman commented 4 years ago

This looks nice!

Having a database backed datastore is very good. Also the key-value pair is handy. It would be nice to be able to change the database, so the key-value API would be a good abstraction for the database.

TimonTerBraak commented 4 years ago

I am writing the layer on top on the Python rocksDB API to be able to read/write the nodes of the btree. That seems to work fine. However, the data structure manipulations are not implemented. That should remain in the btree class.

API is then something like this:

key = addNode(self, left = -1, right = -1, aggregation = [])
key = addLeaf(self, samples)
get(self, key)