Closed TheAdamGalloway closed 2 months ago
Hi there - I think there may be an issue with the Azure OpenAI implementation. You may want to try using LiteLLM as a proxy. Configure Zep to use the standard OpenAI integration and LiteLLM to use Azure OpenAI for LLM and embedding inference.
@danielchalef Thanks for your reply. I took a look at the code at pkg/llms/llm_base.go
that is responsible for printing the error to the console. It looks to just check for the existence of cfg.LLM.AzureOpenAIModel.EmbeddingDeployment
which should correspond to the environment variable ZEP_LLM_AZURE_OPENAI_EMBEDDING_DEPLOYMENT
which I have set to the deployment name for the hosted ada embeddings model - but the console log persists and causes zep to fail.
I can try LiteLLM but it looks like either a bug in the zep code or I have misconfigured somehow? FWIW, zep has served us well and the only problems we have really had have been around environment config - a bit of clarity in the docs would go a long way. Really appreciate your hard work!
@TheAdamGalloway I've committed an Azure OpenAI config fix. Would you please try the latest zep-cloud
container, version [0.26.0](https://github.com/orgs/getzep/packages/container/zep-cloud/233512546?tag=0.26.0)
and let me know if this works for you.
Closing as dup of #296
@danielchalef I can confirm that the bug fix works. Thanks a lot for your help!
Great to hear!
When trying to use Azure OpenAI, I am getting the error: "embeddings model needs to be provided when using Azure API" in the console. I have tried setting:
ZEP_EXTRACTORS_DOCUMENTS_EMBEDDINGS_SERVICE= local ZEP_EXTRACTORS_MESSAGES_EMBEDDINGS_SERVICE=local ZEP_LLM_AZURE_OPENAI_EMBEDDING_DEPLOYMENT=text-embedding-ada-002
We have a deployment of ada, but I'm happy to use local embeddings, I just need it to work.
Can someone point me to the environment variable I need to set? I can't find anything in the documentation.
To Reproduce Set ZEP_LLM_AZURE_OPENAI_ENDPOINT to an Azure OpenAI endpoint.
Expected behavior Embeddings to work.
Logs fatal msg="embeddings model needs to be provided when using Azure API"
Environment (please complete the following information):