search

nbonamy / witsy

Witsy: desktop AI assistant
Apache License 2.0
21 stars 4 forks source link

Feature-Suggest: RAG and TTS #3

Closed snakeying closed 1 day ago

snakeying commented 2 months ago

This desktop app looks very promising, keep up the great work!

Besides the Feature-Suggest: Alternative OpenAI URL and the option for an external Ollama URL @deliciousbob, I recommend adding RAG and TTS functionalities.

  1. For RAG, I suggest using the all-MiniLM-L6-v2 model (https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It's very compact, occupying less than 50MB even in fp16. Although its dimensionality (384 vs. 1536) is lower compared to OpenAI's text-embedding-ada-002, the all-MiniLM-L6-v2 model performs well in practical applications and is free to use. Of course, I also recommend keeping the OpenAI embedding option available for user preference.

  2. For TTS, I suggest using edge-tts (https://migushthe2nd.github.io/MsEdgeTTS/index.html). It is also free and performs excellently in multilingual scenarios.

nbonamy commented 2 months ago

Was thinking about RAG but still unclear how this would work. I am thinking local to a specific conversation but not necessarily intuitive to remember that this conversation is "RAG enabled" for the user...

Have you put any thoughts into it?

Thanks!

nbonamy commented 1 week ago

Read aloud was added to Witsy!

nbonamy commented 5 days ago

RAG has been added to trunk. You can test it. For the moment it only supports OpenAI embeddings. Check it out if you are still interested. Will polish before releasing.

nbonamy commented 3 days ago

1.12 released with RAG aka Document Repository. Check README. Added Ollama embedding on top of OpenAI ones

nbonamy commented 3 days ago

Check https://x.com/nbonamy/status/1833133569436176583 for a demo

nbonamy commented 1 day ago

1.12.4 should be pretty mature for RAG. Closing this issue. Please open a bug report if you find something!