freedmand / semantra

Multi-tool for semantic search
MIT License
2.49k stars 139 forks source link

Re-run queries with tags on new documents #61

Open ribenamaplesyrup opened 1 year ago

ribenamaplesyrup commented 1 year ago

First off, thanks for building this fantastic tool!

One enhancement I’m considering is allowing users to save queries they have written that contain document tags such that these queries can be reused across future sets of documents. 
 I’m proposing this enhancement because I think building queries from tags (as described in 'Step 6: tagging search results') is a process of optimisation that has value worth persisting. We go through a process of discovery in Semantra figuring out which combination of tags and semantic arithmetic will lead us to the most relevant results. It is easy to copy and paste simple queries across separate document searches but replicating queries with tags on new sets of documents is more involved as these queries reference files (embeddings and PDFs) that would need to be saved somewhere sensible to retrieve them. At present, to replicate a query with tags we would also need to re-tag all the document sections in Semantra. 
 I would love to get your general thoughts on whether or not you think this could be a worthwhile enhancement?

I’ve done some basic exploration so far and been able to export a tagged query’s POST request payload which I can re-use by making new requests to the api/query endpoint. I would ideally then like to open Semantra and actually view the results through the interface (effectively reloading the tagged query results) but I haven’t found a good way of doing this.

freedmand commented 10 months ago

I think this is a promising idea! One promising approach may be to: