search

v2rockets / Loyal-Elephie

Your Trusty Memory-enabled AI Companion - Multilingual RAG chatbot optimized for local LLMs | OpenAI API Compatible
MIT License
208 stars 19 forks source link

Question #9

Open hennas-waifson opened 3 weeks ago

hennas-waifson commented 3 weeks ago

Hello! This app is super, I would not even call it an app, it feels alive.

A question about llm settings (sorry, I am a noob). I am running this with command-r-plus, and it works amazingly.

Is there a place to tweak max context length, or does it get passed down from the whatever backend I am using? (Ollama, koboldcpp)

I noticed that some very very long files are possibly not being added to the database in their entirety. Could this be a bug, or maybe there's a limit on file length?

Lastly, a humble ask. Thank you for providing an example of an embeddings server. I am trying to use GPT4 to figure out how to make it work, but no luck so far. I and many many other noobs like myself would appreciate you implementing it fully.

If there's a way I can support the development of this app, please let me know. Thank you!

v2rockets commented 3 weeks ago

I think the first part is similar to the other question https://github.com/v2rockets/Loyal-Elephie/issues/10?

As to embedding server, there is an example code in external_example, feel free to try.

hennas-waifson commented 3 weeks ago

TY for providing that code. I have zero coding experience, but was able to figure it out with aider and gpt-4o, by basically telling it 'to make it work' :-)