search

logan-markewich / bm25-rs

Efficient BM25 indexing using rust
MIT License
13 stars 0 forks source link

Fast changing set #1

Open bm777 opened 1 month ago

bm777 commented 1 month ago

How this work lib works with a dynamic indexed set. Does it do restoring? or Just delete the old index and reindex again.

Now thinking about distributed bm25 :D

logan-markewich commented 1 month ago

You can definitely just insert as new documents come in, no need to reindex. This was actually my main motivation for building this

although right now the index is purely in memory, so you can't quite distribute the actual index itself

(that would likely require a lot of complex logic and sharding)