search

replikativ / konserve

A clojuresque key-value/document store protocol with core.async.
Eclipse Public License 1.0
299 stars 25 forks source link

Hitchhiker-tree backed stores #28

Closed danielsz closed 4 years ago

danielsz commented 4 years ago

It seems that Konserve has the capability to create a store backed by a hitchhiker tree, a feature being put to good use by the Datahike project. However, Konserve doesn't seem to expose this in its store creation API. Would it be possible to lift up a level the calls to have a hitchhiker-tree backed store? Something that Konserve users might use directly? Thank you in advance.

whilo commented 4 years ago

It is the other way around, the hitchhiker-tree can make use of konserve as an IO backend: https://github.com/replikativ/hitchhiker-tree/blob/master/src/hitchhiker/tree/bootstrap/konserve.cljc

I like your idea, I have conceptualized the integration of konserve and the hh-tree as being composable, so you should be able to take the hh-tree and use it for your indexing purposes on top of konserve. How would you like to index your store?

danielsz commented 4 years ago

Nice! Actually, I opened this ticket because I registered to the Scicloj meetup where Konrad presented Datahike, and I skimmed through the code here and there before the meeting. I don't really have a use case, it was a bit whimsical, let's say that Kampbell could use that functonality. Kampbell makes the asumption of a key-value store underneath, does that answer the indexing question?

whilo commented 4 years ago

Yes, it does. If I would have more time I would write a blog post on how to build your own custom database when needed, by plugging together the hitchhiker-tree (or other index structures) on top of konserve with your frontend. I will close this issue for now, feel free to open specific questions of how to use the hitchhiker-tree for kampbell on the hh-tree repo.

danielsz commented 4 years ago

That would be a fascinating blog post indeed. Alas, time is scarce. Thank you for sharing and helping and caring. I will open specific issues when/if relevant. In the meantime, cheers!