search

nextcloud / context_chat_backend

GNU Affero General Public License v3.0
4 stars 5 forks source link

[bug]: OSError: libcufft.so.11: cannot open shared object file #9

Closed scpcom closed 5 months ago

scpcom commented 5 months ago

Describe the bug New version 1.1.0 does not run (reboot loop)

To Reproduce Steps to reproduce the behavior:

  1. Go to 'External Apps'
  2. Click on 'Office'
  3. Click on 'Deploy and Enable' below 'Context Chat Backend'
  4. See error

Expected behavior Container running and usable by Nextcloud.

Context Chat Backend logs (if applicable, from the docker container)

``` OSError: libcufft.so.11: cannot open shared object file: No such file or directory During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/app/main.py", line 55, in create_server(selected_config) File "/app/context_chat_backend/__init__.py", line 35, in create_server model = init_model('embedding', config.get('embedding')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/context_chat_backend/models/__init__.py", line 25, in init_model model = load_model(model_type, model_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/context_chat_backend/models/load_model.py", line 25, in load_model return get_model_for(model_type, model_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/context_chat_backend/models/instructor.py", line 9, in get_model_for return HuggingFaceInstructEmbeddings(**model_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/embeddings/huggingface.py", line 148, in __init__ from InstructorEmbedding import INSTRUCTOR File "/usr/local/lib/python3.11/site-packages/InstructorEmbedding/__init__.py", line 1, in from .instructor import * File "/usr/local/lib/python3.11/site-packages/InstructorEmbedding/instructor.py", line 2, in import torch File "/usr/local/lib/python3.11/site-packages/torch/__init__.py", line 234, in _load_global_deps() File "/usr/local/lib/python3.11/site-packages/torch/__init__.py", line 195, in _load_global_deps _preload_cuda_deps(lib_folder, lib_name) File "/usr/local/lib/python3.11/site-packages/torch/__init__.py", line 160, in _preload_cuda_deps raise ValueError(f"{lib_name} not found in the system path {sys.path}") ValueError: libcublas.so.*[0-9] not found in the system path ['/app', '/usr/local/lib/python311.zip', '/usr/local/lib/python3.11', '/usr/local/lib/python3.11/lib-dynload', '/usr/local/lib/python3.11/site-packages'] ("Selected config: {'vectordb': ('chroma', {'is_persistent': True, " "'persist_directory': './vector_db_data'}), 'embedding': ('instructor', " "{'model_name': 'hkunlp/instructor-base', 'model_kwargs': {'device': " "'cpu'}}), 'llm': ('llama', {'model_path': " "'model_files/dolphin-2.2.1-mistral-7b.Q5_K_M.gguf', 'n_batch': 10, 'n_ctx': " '4096, \'template\': "<|im_start|> system \\nYou\'re an AI assistant good at ' 'finding relevant context from documents to answer questions provided by the ' 'user. <|im_end|>\\n<|im_start|> user\\nUse the following documents as ' 'context to answer the question at the end. REMEMBER to excersice source ' 'critisicm as the documents are returned by a search provider that can return ' 'unrelated documents.\\n\\nSTART OF CONTEXT: \\n{context} \\n\\nEND OF ' "CONTEXT!\\n\\nIf you don't know the answer or are unsure, just say that you " "don't know, don't try to make up an answer. Don't mention the context in " 'your answer but rather just answer the question directly. \\nQuestion: ' "{question} Let's think this step-by-step. \\n<|im_end|>\\n<|im_start|> " 'assistant\\n", \'end_separator\': \'<|im_end|>\'})}') Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/torch/__init__.py", line 174, in _load_global_deps ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL) File "/usr/local/lib/python3.11/ctypes/__init__.py", line 376, in __init__ self._handle = _dlopen(self._name, mode) ^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: libcufft.so.11: cannot open shared object file: No such file or directory During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/app/main.py", line 55, in create_server(selected_config) File "/app/context_chat_backend/__init__.py", line 35, in create_server model = init_model('embedding', config.get('embedding')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/context_chat_backend/models/__init__.py", line 25, in init_model model = load_model(model_type, model_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/context_chat_backend/models/load_model.py", line 25, in load_model return get_model_for(model_type, model_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/context_chat_backend/models/instructor.py", line 9, in get_model_for return HuggingFaceInstructEmbeddings(**model_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain/embeddings/huggingface.py", line 148, in __init__ from InstructorEmbedding import INSTRUCTOR File "/usr/local/lib/python3.11/site-packages/InstructorEmbedding/__init__.py", line 1, in from .instructor import * File "/usr/local/lib/python3.11/site-packages/InstructorEmbedding/instructor.py", line 2, in import torch File "/usr/local/lib/python3.11/site-packages/torch/__init__.py", line 234, in _load_global_deps() File "/usr/local/lib/python3.11/site-packages/torch/__init__.py", line 195, in _load_global_deps _preload_cuda_deps(lib_folder, lib_name) File "/usr/local/lib/python3.11/site-packages/torch/__init__.py", line 160, in _preload_cuda_deps raise ValueError(f"{lib_name} not found in the system path {sys.path}") ValueError: libcublas.so.*[0-9] not found in the system path ['/app', '/usr/local/lib/python311.zip', '/usr/local/lib/python3.11', '/usr/local/lib/python3.11/lib-dynload', '/usr/local/lib/python3.11/site-packages'] ```

Setup Details (please complete the following information):

kyteinsky commented 5 months ago

Wow that was fast! I pushed the build last night. Thanks to the night, I didn't test it and was completely absorbed in making arm releases. I'll revert the pytorch dep change and release a new one.
Thanks for the report!

kyteinsky commented 5 months ago

v1.1.1 released