search

zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks
https://privategpt.dev
Apache License 2.0
53.87k stars 7.24k forks source link

ImportError: Could not import llama_cpp library #1276

Closed kevXZY closed 10 months ago

kevXZY commented 10 months ago

OS: Ubuntu 22.04.3 LTS ARM 64bit using VMware fusion on Mac M2.

I installed LlamaCPP and still getting this error:

~/privateGPT$ PGPT_PROFILES=local make run poetry run python -m private_gpt 02:13:22.418 [INFO ] private_gpt.settings.settings_loader - Starting application with profiles=['default', 'local'] Traceback (most recent call last): File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/init.py", line 798, in get return self._context[key]


KeyError: <class 'private_gpt.ui.ui.PrivateGptUi'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 798, in get
    return self._context[key]
           ~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.server.ingest.ingest_service.IngestService'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 798, in get
    return self._context[key]
           ~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.components.llm.llm_component.LLMComponent'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/llama_index/llms/llama_cpp.py", line 88, in __init__
    from llama_cpp import Llama
ModuleNotFoundError: No module named 'llama_cpp'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/kev/privateGPT/private_gpt/__main__.py", line 5, in <module>
    from private_gpt.main import app
  File "/home/kev/privateGPT/private_gpt/main.py", line 11, in <module>
    app = create_app(global_injector)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/privateGPT/private_gpt/launcher.py", line 125, in create_app
    ui = root_injector.get(PrivateGptUi)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 91, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 974, in get
    provider_instance = scope_instance.get(interface, binding.provider)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 91, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 800, in get
    instance = self._get_instance(key, provider, self.injector)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 811, in _get_instance
    return provider.get(injector)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 264, in get
    return injector.create_object(self._cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 998, in create_object
    self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 1031, in call_with_injection
    dependencies = self.args_to_inject(
                   ^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 91, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 1079, in args_to_inject
    instance: Any = self.get(interface)
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 91, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 974, in get
    provider_instance = scope_instance.get(interface, binding.provider)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 91, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 800, in get
    instance = self._get_instance(key, provider, self.injector)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 811, in _get_instance
    return provider.get(injector)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 264, in get
    return injector.create_object(self._cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 998, in create_object
    self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 1031, in call_with_injection
    dependencies = self.args_to_inject(
                   ^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 91, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 1079, in args_to_inject
    instance: Any = self.get(interface)
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 91, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 974, in get
    provider_instance = scope_instance.get(interface, binding.provider)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 91, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 800, in get
    instance = self._get_instance(key, provider, self.injector)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 811, in _get_instance
    return provider.get(injector)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 264, in get
    return injector.create_object(self._cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 998, in create_object
    self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/injector/__init__.py", line 1040, in call_with_injection
    return callable(*full_args, **dependencies)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kev/privateGPT/private_gpt/components/llm/llm_component.py", line 20, in __init__
    self.llm = LlamaCPP(
               ^^^^^^^^^
  File "/home/kev/.cache/pypoetry/virtualenvs/private-gpt-f8Ius0_a-py3.11/lib/python3.11/site-packages/llama_index/llms/llama_cpp.py", line 90, in __init__
    raise ImportError(
ImportError: Could not import llama_cpp library.Please install llama_cpp with `pip install llama-cpp-python`.See the full installation guide for GPU support at `https://github.com/abetlen/llama-cpp-python`
lopagela commented 10 months ago

@kevXZY

I would recommend you to install directly on your mac (this is safe given that the installation will be done in a python venv, and that there are no system library required for macs if you are using metal)

Performances of VMs are not great, especially when the VM system emulate generic CPUs.

We are currently working on refactoring the documentation, but in a nutshell, to install this on macos:

# One liner to install
git clone https://github.com/imartinez/privateGPT && cd privateGPT && \
python3.11 -m venv .venv && source .venv/bin/activate && \
pip install --upgrade pip poetry && poetry install --with ui,local && ./scripts/setup

# Patch the version of `llama-cpp-python` installed to add metal support
# This have to run with inside your python venv !
CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python

c.f. https://privategpt-preview-8380d3f2-962a-400e-87d4-fd1c4636f174.docs.buildwithfern.com/overview/welcome/quickstart (preview of documentation)

If you still want and persist in VM and Ubuntu world - I guess you simply have to install the missing package, as the error states. (But again, it will not take full advantage of your M2 chips with the chips that apple especially designed for neural network computation)