khoj-ai / khoj

Your AI second brain. Self-hostable. Get answers from the internet or your docs. Use any online or local LLM (e.g gpt, claude, gemini, llama, qwen, mistral). Build custom agents, personalized automations.
https://khoj.dev
GNU Affero General Public License v3.0
12.83k stars 656 forks source link

help text on improve search results #207

Closed whatSocks closed 1 year ago

whatSocks commented 1 year ago

what are some strategies for improving search results? For some more popular concepts, it seems like the search results are good - but very frequently if I use the search for the title of a note it doesn't find it, or any notes related to it

debanjum commented 1 year ago

Umm, I recently updated khoj to index the title/filename of markdown note in https://github.com/debanjum/khoj/commit/a2ab68a7a2f0d7e0397051a54b5fd67a4052d907. Can you share which version of khoj you are on (i.e execute khoj --version in terminal) ? Are using khoj with markdown/obsidian?

debanjum commented 1 year ago

To improve search quality you can try:

  1. Use natural language queries that are longer than the usual keyword based search queries . E.g "visited national park, saw animals" vs "national park cheetah"
  2. Rerank the results after you've entered your query in the Khoj Search Modal on Obsidian (with Cmd+Enter or Ctrl+Enter). This should improve ranking quality of results (it uses a slower but better model for reranking)
  3. Add word, date or file filters to the query to filter results. See Query Filters under Khoj search section in Readme for usage
whatSocks commented 1 year ago

@debanjum turns out i had to re-index, I hadn't gotten to the "results are wonky" part of the readme. Would it be possible to add that suggestion to the search results as a help text? If I hadn't been checking out the repo at the same time as searching I would have given up

debanjum commented 1 year ago

Ah, good to know re-indexing your Vault fixed the issue. There is some intermittent index corruption issue, which I've not been able to reproduce and hence fix yet. Seems like it may have affected you.

Good idea on finding a more direct way of notifying users. Can you expand on what you mean by adding help text?

whatSocks commented 1 year ago
Screenshot 2023-04-08 at 12 52 04 PM

Maybe here under the search bar put

results not what you expected? Try re-indexing your vault."

debanjum commented 1 year ago

I see, thanks for the idea! Let me see what can be done to suggest re-indexing vault if search results are not good anymore