MatrixAI / js-db

Key-Value DB for TypeScript and JavaScript Applications
https://polykey.com
Apache License 2.0
5 stars 0 forks source link

Integrate DB into the level db interface so db levels are the same as db #8

Closed CMCDragonkai closed 2 years ago

CMCDragonkai commented 2 years ago

Specification

Similar to #5, I find that it shouldn't be necessary in PK to have db and vaults running their own DB. The vaults is using EFS which has to use the root DB, but the operations should be workable under a sublevel of the DB used by all of PK.

To do this, the API of js-db has to be brought into the leveldb abstract interface, that creation of sublevels is the same as creating another DB.

Another way is to just make our level call return another DB instance, but always with the option of accessing the underlying .db property for the underlying leveldb to do certain operations.

This should be done with investigating #5 so that a consistent solution can be found.

Additional context

Tasks

  1. ...
  2. ...
  3. ...
CMCDragonkai commented 2 years ago

The way subleveldown works is that it forces prefixes to be strings, even though they could be buffers instead. It also uses the ! as the default prefix separator, although this can be changed. This is something I'd like changed, so that sublevels can be created with buffers purely rather than strings.

One of the things that could be done is to allow an interface like:

get(keys: string | Buffer | Array<string | Buffer>)

This would enable us to do things like:

get('some key')
get(['prefix1', 'prefix2', Buffer.from('prefix3'), 'abc'])

Which is alot more flexible than the current interface we have.

Right now some special features we want to use in leveldb can only be done with db.db.

Instead our DB is currently wrapping db in order to provide a slightly different interface for our usage, and the automatic encryption/decryption (which according to #5 would be better done on a lower level, below the leveldb level).