FR Enhanced Filtering

[chrisfurlong-aiwyn]

Agnostic RAG is obviously a difficult problem, more difficult than implementing it for a specific usecase or domain. The more we can deterministically filter embeddings from consideration, the better it'll work.

I suggest enhancing the filtering technology to move beyond restricting on a per note / directory basis, but also filtering based on tag, note created/updated time, and any other metadata that available to you. I think this is applicable to both chat and to the display of "similar content", and if you support automated note linking one day, I'd suggest a way to respect filters for that too (though maybe it's more dynamic based on DataView)

Bonus points if we can save these combinations, but I don't think it's necessary day 1.

[brianpetro]

Hey @chrisfurlong-aiwyn

This is something being worked on in v2.2 (currently in early-release testing with supporters, screenshot below).

You can see include/exclude filters, which should handle the note/directory filtering you mentioned.

The re-ranker is useful for generally improving the results. A local model version of the re-ranker will be available when the feature lands on the main stable branch.

Tag filtering might be achievable using something like a keyword filter. I haven't thought much about it, but that could be a useful addition to the Smart View filter.

Filtering content retrieved for the Smart Chat could be a little trickier. I suppose the same/similar filter UI could be added to the chat. I also want to add an additional UI to the chat which would allow confirmation (unselecting irrelevant context) prior to submitting the retrieved context for the final chat output.

While I think it may be out of the scope of this request specifically, I have been thinking about saving variations of the Smart View. I have some Smart View adjacent ideas that I think would be better alternatives to the default Smart View, and some that would be good in specific situations. This may end up looking like presets, but also possibly saving various Smart Views that can be opened alongside each other, presenting different frames of reference in each.

Thanks for the feature request 😊🌴