search

getzep / zep

Zep | The Memory Foundation For Your AI Stack
https://help.getzep.com/ce
Apache License 2.0
2.7k stars 384 forks source link

quirks with installing multilingual embeddings model in zep-nlp #369

Closed kilmarnock closed 1 month ago

kilmarnock commented 1 month ago

I use a docker container install. It boots up as expected. When I pass in jinaai/jina-embeddings-v2-base-de as ZEP_EMBEDDINGS_MESSAGES_MODEL, it tries to execute some custom code (as intended), the error message that shows up is

ValueError: The repository for /root/.cache/torch/sentence_transformers/jinaai_jina-embeddings-v2-base-de/ contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co//root/.cache/torch/sentence_transformers/jinaai_jina-embeddings-v2-base-de/.

Please pass the argument trust_remote_code=True to allow custom code to be run.

2024-09-27 12:00:05,326 - uvicorn.error - ERROR - Application startup failed. Exiting.

I tried to pass in trust_remote_code=True as environment variable and as COMMAND in the docker-compose file, without succes.

When I pass in jinaai/jina-embeddings-v3 as desired multilingual model, it gives me

You try to use a model that was created with version 3.1.0, however, your version is 2.2.2. This might cause unexpected behavior or errors. In that case, try to update to the latest version. and ModuleNotFoundError: No module named 'custom_st'

Please, do you see a shortcut to pass in the trust_remote_code=True parameter?

(Besides, your webserver is on a "redirect every link to the same page" trip today.)

danielchalef commented 1 month ago

Hi there - we're unable to support v0.x of Zep Open Source and the related NLP service is deprecated.

The documentation may now be found here: https://help.getzep.com/ce/legacy/deployment/quickstart

The parameter you mentioned needs to be added to the model loader code, not as an argument to the container.

The NLP service uses outdated versions of inference libraries and its possible- given the second error message you provided - that the model won't load successfully, even with the code change.

I recommend investigating the recently released Zep Community Edition. It has a simpler setup, is more accurate, and flexible. With a proxy such as LiteLLM, Zep CE is able to connect to many embedding services, including locally hosted.

kilmarnock commented 1 month ago

Hi Daniel, thank you for your fast reply. I will check the community edition. Wow, you released the ce yesterday, congratulations!

danielchalef commented 1 month ago

Thank you!