search

getmetal / motorhead

🧠 Motorhead is a memory and information retrieval server for LLMs.
https://getmetal.io
Apache License 2.0
843 stars 79 forks source link

More details on why long-term doesn't work with Azure deployment #78

Closed tuanalumi closed 9 months ago

tuanalumi commented 11 months ago

I see this note in readme

NOTE: MOTORHEAD_LONG_TERM_MEMORY=true won't work with Azure.

But it's unclear to me if the long-term feature doesn't work at all, or just the MOTORHEAD_LONG_TERM_MEMORY config. And why is that so?

The way I understand this, is that long-term memory uses Redisearch VSS to store the chat history. So it's just the matter of whether the Redis instance has Redisearch feature available. What do I miss here?

Czechh commented 11 months ago

Hey @tuanalumi - thanks for bringing this up.

At the time, I wasn't sure if you could have both an LLM and an embedding model in a single Azure deployment. It looks like that is the case? Can you confirm, please.

We can adapt to allow this, just need to make sure that the azure client gets used for the embedding generation.

daviddahl commented 11 months ago

I can confirm this. I have ada and gpt4 in the same deployment sharing an API key and endpoint.

Czechh commented 11 months ago

Thank you @daviddahl - we'll get to this this week, otherwise PRs welcome!

paulpierre commented 9 months ago

Hi was this resolved?

shubham184 commented 9 months ago

+

Czechh commented 9 months ago

I see that the code is there to generate the embeddings using azure as well, so I think that note might've been more of a warning on my end. Has anyone tested this? I think it might just work.

Czechh commented 9 months ago

I just added AZURE_DEPLOYMENT_ID_ADA to fix this. Will release soon, it will be a breaking change though