search

hadynz / obsidian-sidekick

A companion to identify hidden connections that match your tags and pages
134 stars 9 forks source link

Curious why lokijs and what it is doing? Thanks! #20

Closed GitMurf closed 2 years ago

GitMurf commented 2 years ago

I am just curious why you chose to use lokiJS database instead of just using a global JS variable arrays/objects? Given that lokiJS doesn't persist to disk and the index is re-built each time on Obsidian load, is it doing any "magic sauce" that a native JS array/object couldn't do? Or is it just something you are familiar with using so you like to use it?

I just am kind of OCD about having plugins add packages/libraries and love to understand why (also to learn).

Thanks! The plugin is coming along nicely!

hadynz commented 2 years ago

To optimise for performance I needed to:

This meant that I needed a data structure to store the index that I can access in two ways:

The above requirements meant that using a simple global array or object won't perform well as I'd need to a full scan for one of the use cases. The other option I have is to have multiple arrays or objects that I keep in sync to fetch the data I need efficiently.

That's when I realised that I'm heading down a direction were I will be building a DB engine.

For the above reasons I started considering using an in-memory database that allows me to efficiently query the data in many different ways in a fast and performant manner.

Also, given that the plug-in is still in active development I don't know what all my use cases will end up being. My big bet is that using something like LokiJs will allow me to easily iterate and innovate more quickly then building rigid bespoke structures.

Lastly. LokiJs has the option to save its in-memory to disk. That would help tremendously as it means the index is only built once when the plug-in is enabled only. I haven't turned on this feature yet as I'm prioritising fixing bugs and adding feature at the moment.

GitMurf commented 2 years ago

@hadynz sorry I never responded to this... GitHub notifications don't do a good job at notifying me haha. Thanks for the explanation! That is what I was looking for. And this is also great: "Lastly. LokiJs has the option to save its in-memory to disk."