olivernn / lunr.js

A bit like Solr, but much smaller and not as bright
http://lunrjs.com
MIT License
8.87k stars 546 forks source link

[Feature Request] Allow documents to be added/removed after index creation #531

Open jasonpolites opened 3 weeks ago

jasonpolites commented 3 weeks ago

After reviewing the docs, and some older issues it became clear that any documents in the index need to be added during index creation. A quick look at the source code suggests that much of the index structure assumes some stability in the document set (to calculate field vectors etc), so this requirement makes.

However it implies a significant drawback as a key use case is prevented by this approach.

In circumstances where the documents to be added to the index may be created during a user session, the entire index would need to be rebuilt every time. For small numbers of documents this might be ok, but in large numbers this may mean this approach is untenable.

In these situations my intuition is that a preferred approach would be to persist the index state to disk (or indexeddb) and operate on it incrementally during each session. This would imply adding an add and delete method to the Index object.

Looking at the code this seems like it would be a non-trivial change, so I suspect it may be unlikely to be done (if it were simple, I likely would have considered contributing a PR to add it, but I suspect it would require a fairly large rewrite). However as simple and well written as lunr seems to be (the "simple" phrasing is a compliment, the code is very readable), I am unable to use it without the ability to operate on an Index incrementally.

It's a shame, because based on my research, lunr is the best solution for a lightweight, client-side inverted index. Thanks for contributing to the community in this way :)