dosyago / sirdb

:man: a simple, git diffable JSON database on yer filesystem. By the power of NodeJS
GNU Affero General Public License v3.0
572 stars 14 forks source link

indexes #6

Closed o0101 closed 4 years ago

o0101 commented 4 years ago

let's build some simple indexes....

aid in property uniqueness queries, and other queries.

let's come up with a nice way to do these

(at sd or studdb level?)

o0101 commented 4 years ago

probably at sd level, since type validation of inputs happens here.

eventually me might be able to shift more and more DB related stuff to stubdb...but for now it's wrapped up here.

o0101 commented 4 years ago

for now it's wrapped up here.

which is fine

o0101 commented 4 years ago

Stubdb can do indexes.

So when we create a table we can specify an index:[] option, that lists properties that are to be indexed.

Then everytime we get a write action (newItem/setItem) what happens is this, the index directory is found

/<table_name>/_indexes/<property_name>

and the value of the property to be indexes is hashed, to give <value_hash> and the file:

/<table_name>/_indexes/<property_name>/<value_hash>.json

is found or created. And the record id is added to the index record for that value. If there's a collision, the record also adds the collided value.

An index record looks like this:

{
   [base64value]: [<record_ids>],
   [base64value]: [<record_ids>],
}

so any collisions are handled like that.

Then in order to retrieve results for an index, we simply hash the value and get the index record, and we have the results right there.