Open jasmeetchadha opened 1 year ago
Thanks for this write-up. Pretty spot-on in terms of things I'd like to tackle soon. Will address some thoughts for each:
semantra --show-semantra-dir
to see where they are all stored, or semantra --semantra-dir [folder]
to change it.I get your hesitance with summarisation. I think highlighting source information alongside a well written summary is pretty cool. My point is more to your point of keeping things local. And to get that well written "completion" using a local copy of Vicuna iso OpenAI
Funny enough I found a repo that does it and demoed here: https://vault.pash.city/ https://github.com/pashpashpash/vault-ai
The local LLMs idea is definitely intriguing. An additional layer you could potentially utilize with local models: visually showing the model's attention over parts of the summary. You could potentially click on parts of the summary and see which search result context is most utilized to inform each part of the summary. I think I'd only be interested in summarization if we could take it a step further like this with more explainability built-in 😃
@jasmeetchadha You might be interested in LocalGPT, which also feeds embeddings results to a local LLM. It requires more resources than my local machines can handle, though, and I don't trust the output of LLMs. Semantra looks more promising for the kinds of things I want to do, finding verbatim citations by concept, etc.
https://github.com/PromtEngineer/localGPT/discussions/186
I would sometimes want to feed the output to an LLM, though, to find an answer to a question that requires combining multiple result chunks of technical text, for instance.
I also plan to use this to search documents in other languages or multiple languages in one file, so it would be good if I could see a translation of the citation into English alongside the quote instead of having to copy and paste into a translator. So some general thing that makes it easier to pipe the results into other tools would be nice. (But I haven't even tried Semantra, yet.)
I like the idea of semantic search. To expand on this (and I am sure it has been said before). It would be good to add:
UI / Input 1) Options to interact from WebUI iso command line. Adding files etc 2) Increased filetype coverage - include office, markdown, images (?) 3) Input a folder - with all files covered
Processing 4) Specify where embeddings are stored from UI (or make it clear in readme) 5) Save embeddings to avoid repeat processing 5) Auto-update embedding at startup for any new files that may have been added to folder 6) Enable GPU mode
Summarization (additional) 7) Perhaps use Vicuna (quite good already) or another local LLM to process the results from the search query into a coherent reply (with sources, links to source document). 8) This reply can be adjacent to a "source window" (in case of multiple documents), keeping full visibility into source text for the user.