search

getzep / zep

Zep: Long-Term Memory for ‍AI Assistants.
https://docs.getzep.com
Apache License 2.0
2.24k stars 346 forks source link

Any good open source LLMs for zep's summarization? #300

Open Elsayed91 opened 8 months ago

Elsayed91 commented 8 months ago

This is not a feature/bug, just a question, since there is no discussion board I'll leave it here.

I have multiple hugging face models hosted and running OpenAI compatible endpoints. After hooking them up to Zep, they continue to work, however the results were kind of disappointing for the models I have tried. They get stuck in the whole Led Zeppelin part of the default prompt template, the intention extraction starts spinning up weird things and gets stuck up on Nike.

Has anyone used any huggingface LLMs for this with any degree of success? If so, please provide name and template you used if you changed the default template.

PS: It would be nice to be able to change intent prompt template in order to make it match the prompting style of selected model. (Someone has raised this already)

danielchalef commented 7 months ago

We're in the middle of a significant rewrite of Zep's LLM use, with far more flexibility around the use of different models. This will be released towards the end of the quarter.

cannuri commented 2 months ago

any news on this?