Closed snakeying closed 1 day ago
Was thinking about RAG but still unclear how this would work. I am thinking local to a specific conversation but not necessarily intuitive to remember that this conversation is "RAG enabled" for the user...
Have you put any thoughts into it?
Thanks!
Read aloud was added to Witsy!
RAG has been added to trunk. You can test it. For the moment it only supports OpenAI embeddings. Check it out if you are still interested. Will polish before releasing.
1.12 released with RAG aka Document Repository. Check README. Added Ollama embedding on top of OpenAI ones
Check https://x.com/nbonamy/status/1833133569436176583 for a demo
1.12.4 should be pretty mature for RAG. Closing this issue. Please open a bug report if you find something!
This desktop app looks very promising, keep up the great work!
Besides the Feature-Suggest: Alternative OpenAI URL and the option for an external Ollama URL @deliciousbob, I recommend adding RAG and TTS functionalities.
For RAG, I suggest using the all-MiniLM-L6-v2 model (https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It's very compact, occupying less than 50MB even in fp16. Although its dimensionality (384 vs. 1536) is lower compared to OpenAI's text-embedding-ada-002, the all-MiniLM-L6-v2 model performs well in practical applications and is free to use. Of course, I also recommend keeping the OpenAI embedding option available for user preference.
For TTS, I suggest using edge-tts (https://migushthe2nd.github.io/MsEdgeTTS/index.html). It is also free and performs excellently in multilingual scenarios.