More lenient date searching

[Jonny-GM]

Suppose we have a scenario where we have notes such that the date of the note is in the filename; this is prevalent and is for example the default behavior of the Obsidian Daily Notes feature. Chatting using a simple chat message such as "what does yesterday's note say" then fails to give relevant results, for 2 reasons:

1. By default the DateFilter uses the raw content of the entry, rather than the compiled content which includes the filename. Given how prevalent naming notes by date is, I think the DateFilter should use the compiled content.
2. The defiltered search query is then something like "Note from yesterday" which is below the score threshold because the note contents itself generally contains nothing about "yesterday" or "note". Information about the actual date should be included (what's the reasoning behind defiltering the search query in the first place? keeping privacy about the full file-path?)

This PR resolves these such that the simple chat query works as expected.

[debanjum]

Thanks for the thorough thinking for this PR! 👌🏾

what's the reasoning behind defiltering the search query in the first place? keeping privacy about the full file-path?)

• The idea to defilter the search query once the filtered results have been retrieved from your knowledge base is because:

  1. The Search model can just try find the most relevant of the filtered notes using the defiltered query, and not get potentially confused by the query filter format
  2. The Chat models used to get confused by the query filters. Given all the references passed to it have been explicitly filtered to the relevant set already, defiltering the user query helped chat responses.

  This logic is unfortunately mostly based on ad-hoc testing rather than proper benchmarks. And it will create issues when you ask general questions like "what does yesterday's note say" for the exact reasons you inferred.

• Looking at filename for the date filters makes sense

• Passing the user query with filters to chat model needs to be tested out. The larger models may not be thrown of by the query filter format now. We can run the Khoj chat quality tests to see if it doesn't degrade its capabilities