Open newmonte opened 9 months ago
Can you write here the full list of command or operation that you've made for your local installation please?
I guess you have read the installation guide from the documentation https://docs.privategpt.dev/installation, and that you also run the set CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
Thank you Lopagela,
I followed the installation guide from the documentation, the original issues I had with the install were not the fault of privateGPT, I had issues with cmake compiling until I called it through VS 2022, I also had initial issues with my poetry install, but now after running set CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
privateGPT is now working, I had seen this command but mistakenly assumed it was optional and only needed to run it if I wanted to use gpu.
Thankyou very much for your help.
I was facing similar issues.
set CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
and came up empty.make run
, and it would fail with this message:
File "C:\Users\me\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-TFCUF6yI-py3.11\Lib\site-packages\llama_index\llms\llama_cpp.py", line 119, in init raise ValueError( ValueError: Provided model path does not exist. Please check the path or provide a model_url to download. make: *** [Makefile:36: run] Error 1
Whereas the installation guide says that at this point, I should be able to visit the UI in-browser.
poetry install --with local
and poetry run python scripts/setup
. At that point, I was able to use the UISo I think that the installation guide might be mistaken, have missing parts, be out of date, or at least not cover some configurations or environments.
I also tend to think that some steps are missing in the installation guide.
After finishing the chapter 'Install dependencies' I expected to get working UI with a mock LLM, but executing poetry run python -m private_gpt
ends with the errors:
...
File "C:\Temp\privateGPT\Lib\site-packages\injector\__init__.py", line 998, in create_object
self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
File "C:\Temp\privateGPT\Lib\site-packages\injector\__init__.py", line 1040, in call_with_injection
return callable(*full_args, **dependencies)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Temp\privateGPT\private_gpt\components\llm\llm_component.py", line 28, in __init__
self.llm = LlamaCPP(
^^^^^^^^^
File "C:\Temp\privateGPT\Lib\site-packages\llama_index\llms\llama_cpp.py", line 119, in __init__
raise ValueError(
ValueError: Provided model path does not exist. Please check the path or provide a model_url to download.
One step is certainly missing at this point - installing llama-cpp-python
:
pip install llama-cpp-python==0.2.18
At this point the subdir models
is usually empty, and it seems some model are expected to be found there.
According to the guide the models are downloaded later (with a command like poetry run python scripts/setup
).
same, followed the guide but with one exception:
devbox global add pyenv
devbox global shell
... now follow rest of guide.
and i had to finally do this:
poetry add uvicorn
otherwise i get
privateGPT on git main is pkg v0.1.0 via py v3.11.4 via nix impure (nix-shell-env) took 14s
> poetry run python -m private_gpt
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/mnt/Store/Projects/Others/privateGPT/private_gpt/__main__.py", line 3, in <module>
import uvicorn
ModuleNotFoundError: No module named 'uvicorn'
woops, not the end of it -_-
poetry run python -m private_gpt
17:10:52.923 [INFO ] private_gpt.settings.settings_loader - Starting application with profiles=['default']
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/mnt/Store/Projects/Others/privateGPT/private_gpt/__main__.py", line 5, in <module>
from private_gpt.main import app
File "/mnt/Store/Projects/Others/privateGPT/private_gpt/main.py", line 11, in <module>
app = create_app(global_injector)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/Store/Projects/Others/privateGPT/private_gpt/launcher.py", line 48, in create_app
from private_gpt.ui.ui import PrivateGptUi
File "/mnt/Store/Projects/Others/privateGPT/private_gpt/ui/ui.py", line 8, in <module>
import gradio as gr # type: ignore
^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'gradio'
make: *** [Makefile:36: run] Error 1
rerunning:
privateGPT on git main [!] is pkg v0.1.0 via py v3.11.4 via nix impure (nix-shell-env) took 6s
x poetry install --with ui
solved it.
@airtonix , the modules like gradio
or uvicorn
are usually installed when executing poetry install --with ui
. Anyway that's great you've solved your issue on your own! :)
The error like ValueError: Provided model path does not exist. Please check the path or provide a model_url to download.
when running LLM in so called mock-mode (reference) usually means that settings.yaml
is incorrect: by default it specifies mode: local
for llm
(which isn't set up unless you executed poetry install --with local
earlier), so changing this setting to mode: mock
can help you to resolve this issue, and ... reach another one when you open a browser and send any message to LLM:
Traceback (most recent call last):
File "C:\Temp\privateGPT\Lib\site-packages\gradio\queueing.py", line 501, in process_events
response = await self.call_prediction(awake_events, batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Temp\privateGPT\Lib\site-packages\gradio\queueing.py", line 465, in call_prediction
raise Exception(str(error) if show_error else None) from error
Exception: Collection make_this_parameterizable_per_api_call not found
As proposed in https://github.com/imartinez/privateGPT/issues/1334#issuecomment-1847997849 you should tune your settings.yaml
one more time - change database: qdrant
(for vectorstore
) with database: chroma
, and execute poetry install --extras chroma
(documentation).
And this is what I finally get in mock-mode:
I'm not sure it works as expected (I guess, it should echo messages), but at least it doesn't crash :)
Stale issue
This is not a stale issue: I'm experiencing similar issue and couldn't solve applying the suggestions given up to now. This is what happens: make run poetry run python -m private_gpt The currently activated Python version 3.10.0 is not supported by the project (>=3.11,<3.12). Trying to find and use a compatible version. Using python3 (3.11.5) 20:16:34.938 [INFO ] private_gpt.settings.settings_loader - Starting application with profiles=['default'] 20:16:39.821 [INFO ] private_gpt.components.llm.llm_component - Initializing the LLM in mode=local Traceback (most recent call last): File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector__init__.py", line 798, in get return self._context[key]
KeyError: <class 'private_gpt.ui.ui.PrivateGptUi'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 798, in get
return self._context[key]
~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.server.ingest.ingest_service.IngestService'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 798, in get
return self._context[key]
~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.components.llm.llm_component.LLMComponent'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\llama_cpp\llama_cpp.py", line 74, in _load_shared_library
return ctypes.CDLL(str(_lib_path), **cdll_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\.pyenv\pyenv-win\versions\3.11.1\Lib\ctypes\__init__.py", line 376, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 193] %1 non è un'applicazione di Win32 valida
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "C:\Users\M\source\repos\privateGPT\private_gpt\__main__.py", line 5, in <module>
from private_gpt.main import app
File "C:\Users\M\source\repos\privateGPT\private_gpt\main.py", line 11, in <module>
app = create_app(global_injector)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\source\repos\privateGPT\private_gpt\launcher.py", line 50, in create_app
ui = root_injector.get(PrivateGptUi)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 974, in get
provider_instance = scope_instance.get(interface, binding.provider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 800, in get
instance = self._get_instance(key, provider, self.injector)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 811, in _get_instance
return provider.get(injector)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 264, in get
return injector.create_object(self._cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 998, in create_object
self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 1031, in call_with_injection
dependencies = self.args_to_inject(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 1079, in args_to_inject
instance: Any = self.get(interface)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 974, in get
provider_instance = scope_instance.get(interface, binding.provider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 800, in get
instance = self._get_instance(key, provider, self.injector)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 811, in _get_instance
return provider.get(injector)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 264, in get
return injector.create_object(self._cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 998, in create_object
self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 1031, in call_with_injection
dependencies = self.args_to_inject(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 1079, in args_to_inject
instance: Any = self.get(interface)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 974, in get
provider_instance = scope_instance.get(interface, binding.provider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 800, in get
instance = self._get_instance(key, provider, self.injector)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 811, in _get_instance
return provider.get(injector)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 264, in get
return injector.create_object(self._cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 998, in create_object
self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\injector\__init__.py", line 1040, in call_with_injection
return callable(*full_args, **dependencies)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\source\repos\privateGPT\private_gpt\components\llm\llm_component.py", line 38, in __init__
self.llm = LlamaCPP(
^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\llama_index\llms\llama_cpp.py", line 102, in __init__
from llama_cpp import Llama
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\llama_cpp\__init__.py", line 1, in <module>
from .llama_cpp import *
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\llama_cpp\llama_cpp.py", line 87, in <module>
_lib = _load_shared_library(_lib_base_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\llama_cpp\llama_cpp.py", line 76, in _load_shared_library
raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
RuntimeError: Failed to load shared library 'C:\Users\M\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-mWPpvHPQ-py3.11\Lib\site-packages\llama_cpp\llama.dll': [WinError 193] %1 non è un'applicazione di Win32 valida
make: *** [Makefile:36: run] Error 1
same, followed the guide but with one exception:
devbox global add pyenv devbox global shell ... now follow rest of guide.
and i had to finally do this:
poetry add uvicorn
otherwise i get
privateGPT on git main is pkg v0.1.0 via py v3.11.4 via nix impure (nix-shell-env) took 14s > poetry run python -m private_gpt Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/mnt/Store/Projects/Others/privateGPT/private_gpt/__main__.py", line 3, in <module> import uvicorn ModuleNotFoundError: No module named 'uvicorn'
how did you solve it? i'm having this problem with the uvicorn. Use: poetry add uvicorn but the probem stills here
I hadn't solved it in that moment. But now some days ago a new version of privateGPT has been released, with new documentation, and it uses ollama instead of llama. Try with the new version. At most you could use a docker, instead. Join the discord group for updates.
Hello, after a huge amount of trial and error I got this installed in Win 10 but when I "make run" I get the following, I have tried to troubleshoot, (well google the errors) and was lead back here where there is a similar issue reported. Have re installed multiple times now with same result. Any help gratefully received. Thank you.
`C:\Users\J\privateGPT>poetry run python -m private_gpt 17:42:50.705 [INFO ] private_gpt.settings.settings_loader - Starting application with profiles=['default'] 17:42:53.340 [INFO ] private_gpt.components.llm.llm_component - Initializing the LLM in mode=local Traceback (most recent call last): File "C:\Users\J\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-wYPNE1R7-py3.11\Lib\site-packages\injector__init__.py", line 798, in get return self._context[key]