loupe-php / loupe

A full text search engine with tokenization, stemming, typo tolerance, filters and geo support based on only PHP and SQLite.
MIT License
271 stars 8 forks source link

Improve indexing performance #34

Closed Toflar closed 1 year ago

Toflar commented 1 year ago

I've found a few ways to considerably improve indexing performance and basically cut it in half.

One fix was also found in https://github.com/Toflar/state-set-index.

Approx. 30% is now spent on detecting languages which we can improve as well. I did one PR (https://github.com/patrickschur/language-detection/pull/54) but there is probably more potential in that library than just this.