search

jameslittle230 / stork

🔎 Impossibly fast web search, made for static sites.
https://stork-search.net
Apache License 2.0
2.73k stars 56 forks source link

indexation is pretty slow #340

Open setop opened 1 year ago

setop commented 1 year ago

I built an index of 27k entries that are short sentences (nine English words average).

The search is pretty quick : 300ms in CLI, 600ms in the browser.

But building the index has been very long : 15 minutes on a core i7 CPU. Means 30 entries / seconds.

I've been doing the same exercise with other search engine also written is Rust and it was running at 2700 entries / seconds.

Any idea of what cost so much ? Any ways to optimize it ?

jameslittle230 commented 1 year ago

Hi! I'm sure there are optimizations that could be made to the indexer, but I haven't spent much time optimizing the indexer itself (or investigating what could be optimized) -- I've instead focused more on build artifact size and search speed.

I'll leave this issue open for a bit to see if any optimization ideas crop up.

-James