search

Gozala / datalogia

Library for querying in-memory facts using datalog
5 stars 1 forks source link

feat: async support for persistent facts dbs #22

Closed relves closed 6 months ago

relves commented 6 months ago

Interesting library you have here! Figured I'd see what would be involved to support persistent facts DBs. Thoughts?

Gozala commented 6 months ago

Hey @relves thanks for your interest and a PR! I do have some thoughts about persistent store captured by these interfaces (although second one should be async also

https://github.com/Gozala/datalogia/blob/32e7f5981ea33f719b1c20d22faf12189fe54379/src/api.ts#L296-L302

General idea is that store can be any KV and https://github.com/smallhelm/level-fact-base does pretty good job doing roughly what I have in mind.

That said I am not prepared to make all of that async, my thinking is to introduce IO during query planning, that is send FactsSelector off to the store that can do IO and return matches that then query engine can narrow down locally without any asynchrony

https://github.com/Gozala/datalogia/blob/32e7f5981ea33f719b1c20d22faf12189fe54379/src/api.ts#L283-L287

It may not pan out as I am imagining, but I want to try that approach before turning everything async which I want to avoid because I use this engine to query in memory indexes and introducing async loops would render it less practical.

relves commented 6 months ago

Ah yes that sounds like a more elegant approach! I'm wiring up LMDB, hence the interest. Level makes sense as well. Thanks for taking the time to explain your thoughts @Gozala. Closing...

Gozala commented 5 months ago

Ah yes that sounds like a more elegant approach! I'm wiring up LMDB, hence the interest. Level makes sense as well. Thanks for taking the time to explain your thoughts @Gozala. Closing...

That is super interesting, please consider sharing where you'll end up with LMDB. And if you run into some problems in implementing it.