search

itemsapi / itemsjs

Extremely fast faceted search engine in JavaScript - lightweight, flexible, and simple to use
Apache License 2.0
346 stars 41 forks source link

Persistence of index #140

Open jzakotnik opened 10 months ago

jzakotnik commented 10 months ago

Is there a way to persist an existing itemjs index and reload it again for search later? This would be a great enhancement.

If I see it correctly, the index is built via lunr, so the serialisation might also be possible

cigolpl commented 10 months ago

This is an excellent suggestion. Implementing serialization of the search index could potentially result even a 10x performance improvement during reindexing operations, such as when a user refreshes the page. The serialized index could be stored locally using localStorage or as a file on the server side for persistence.

There's a notable parallel here with how pandas can swiftly load Parquet files, which, despite containing the same data as CSV files, are loaded significantly faster due to their optimized format. I'm curious to see how much of a performance gain we could achieve in this context with JavaScript.

I'd be interested to hear if this aligns with your thoughts or if there's anything you might want to add.