Support exclusion file filters

khoj-ai / khoj

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (e.g gpt, claude, gemini, llama, qwen, mistral).

https://khoj.dev

GNU Affero General Public License v3.0

14.09k stars 702 forks source link

Support exclusion file filters #826

Closed srikary12 closed 3 months ago

srikary12 commented 4 months ago

Overview

Support exclude file filter in user search queries

Details

All of the exclude file filter terms need to be satisfied
Any one of the include file filter terms should be satisfied

Example

Search Query: what happened yesterday? -file:"tasks.org" -file:"work.md" file:"diary.org" file:"journal.org
Behavior: Query will try find relevant notes in any of journal.org or diary.org and not in tasks.org and not in work.md

Closes #728

srikary12 commented 4 months ago

@debanjum Can you check and let me know if it's working fine.

srikary12 commented 4 months ago

@debanjum @sabaimran let me know if there are any issues

srikary12 commented 4 months ago

@debanjum added the case

srikary12 commented 4 months ago

Sure, I'll be working on it.

Thanks.

srikary12 commented 3 months ago

@debanjum @sabaimran fixed the tests, Sorry for the delay.

srikary12 commented 3 months ago

test_notes_search_with_only_filters failed, I'm not sure if it's related to this PR.

debanjum commented 3 months ago

Hey @srikary12, I've tested the PR and pushed some changes. These changes are good to merge!

test_notes_search_with_only_filters failed, I'm not sure if it's related to this PR.

The test failure was because of the changes made in this PR (Django ORM expected a .*.org but query had *.org). I've fixed the regex escape logic (and improved the exclusion filter logic)