search

zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks
https://docs.privategpt.dev
Apache License 2.0
53.06k stars 7.13k forks source link

Privategpt keeps on restarting whenever asked question in UI and it's not stable. #1804

Open Ragavendra-Vigneshwaran-R opened 3 months ago

Ragavendra-Vigneshwaran-R commented 3 months ago

I have installed privateGpt in aws ec instance with type g5.2xlarge(8vCPU, 32Gib Memory).I am using llamacpp as LLM TheBloke/Mistral-7B-Instruct-v0.2-GGUF as LLM model in my privategpt steup. In UI, I have upload files which conatins instruction to train models. After that, I asked some question related to it. But it keeps on restarting and throwing the below error like segmentaion fault(core dumped), connection timed out, etc.... Please help me to stablize the privategpt in my setup.(Sometimes after reloading the browser, the UI can answer the questin even that takes about 40 seconds ar above) Errors:

File "/root/.cache/pypoetry/virtualenvs/private-gpt-Wtvj2B-w-py3.11/lib/python3.11/site-packages/qdrant_client/qdrant_client.py", line 345, in search
    return self._client.search(
           ^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/private-gpt-Wtvj2B-w-py3.11/lib/python3.11/site-packages/qdrant_client/local/qdrant_local.py", line 186, in search
    return collection.search(
           ^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/private-gpt-Wtvj2B-w-py3.11/lib/python3.11/site-packages/qdrant_client/local/local_collection.py", line 393, in search
    name, query_vector = self._resolve_query_vector_name(query_vector)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/private-gpt-Wtvj2B-w-py3.11/lib/python3.11/site-packages/qdrant_client/local/local_collection.py", line 204, in _resolve_query_vector_name
    elif isinstance(query_vector, get_args(QueryVector)):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: isinstance() argument 2 cannot be a parameterized generic
  File "/root/.cache/pypoetry/virtualenvs/private-gpt-Wtvj2B-w-py3.11/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/private-gpt-Wtvj2B-w-py3.11/lib/python3.11/site-packages/gradio/utils.py", line 490, in run_sync_iterator_async
    return next(iterator)
           ^^^^^^^^^^^^^^
  File "/home/ubuntu/privateGPT/private_gpt/ui/ui.py", line 136, in _chat
    all_messages = [*build_history(), new_message]
                     ^^^^^^^^^^^^^^^
  File "/home/ubuntu/privateGPT/private_gpt/ui/ui.py", line 118, in build_history
    *[
     ^
  File "/home/ubuntu/privateGPT/private_gpt/ui/ui.py", line 123, in <listcomp>
    content=interaction[1].split(SOURCES_SEPARATOR)[0],
            ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'split'

Llama.generate: prefix-match hit
/home/ubuntu/privateGPT/startup.sh: line 6: 1303912 Segmentation fault      (core dumped) /home/ubuntu/.local/bin/poetry run python -m private_gpt

Is this issue related to insuffiecient memory? or Do i need to increase my memory for ec2 to stablize the privategpt?

invisiblepancake commented 3 months ago

i have having some buglooops with pyenv and ruff . check pyproject.toml that ruff if 0.2.2 (can just type it in) and just start make run hunting.toml example:

poerty setup/scripts from project root terminal bash might just mix it all together also, after u have all dependancies... if ai keeps uninstalling lokk for extra 1 extra2 etc =)

[tool.poetry] name = "private-gpt" version = "0.4.0" description = "Private GPT" authors = ["Zylon hi@zylon.ai"]

[tool.poetry.dependencies] python = ">=3.10,<3.11" fastapi = {extras = ["all"], version = "^0.110.0"} python-multipart = "^0.0.9" injector = "^0.21.0" pyyaml = "^6.0.1" watchdog = "^4.0.0" transformers = "^4.38.2" llama-index-core = "^0.10.14" llama-index-readers-file = "^0.1.6" llama-index-llms-llama-cpp = {version = "^0.1.3", optional = true} llama-index-llms-openai = {version = "^0.1.6", optional = true} llama-index-llms-openai-like = {version = "^0.1.3", optional = true} llama-index-llms-ollama = {version = "^0.1.2", optional = true} llama-index-llms-azure-openai = {version = "^0.1.5", optional = true} llama-index-embeddings-ollama = {version = "^0.1.2", optional = true} llama-index-embeddings-huggingface = {version = "^0.1.4", optional = true} llama-index-embeddings-openai = {version = "^0.1.6", optional = true} llama-index-embeddings-azure-openai = {version = "^0.1.6", optional = true} llama-index-vector-stores-qdrant = {version = "^0.1.3", optional = true} llama-index-vector-stores-chroma = {version = "^0.1.4", optional = true} llama-index-vector-stores-postgres = {version = "^0.1.2", optional = true} llama-index-storage-docstore-postgres = {version = "^0.1.2", optional = true} llama-index-storage-index-store-postgres = {version = "^0.1.2", optional = true} psycopg2-binary = {version = "^2.9.9", optional = true} asyncpg = {version = "^0.29.0", optional = true} boto3 = {version = "^1.34.51", optional = true} gradio = {version = "^4.19.2", optional = true} pydantic = "^2.6.4" torch = "^2.2.1" torchvision = "^0.17.1" tensorflow = "^2.16.1"

[tool.poetry.extras] ui = ["gradio"] llms-llama-cpp = ["llama-index-llms-llama-cpp"] llms-openai = ["llama-index-llms-openai"] llms-openai-like = ["llama-index-llms-openai-like"] llms-ollama = ["llama-index-llms-ollama"] llms-sagemaker = ["boto3"] llms-azopenai = ["llama-index-llms-azure-openai"] embeddings-ollama = ["llama-index-embeddings-ollama"] embeddings-huggingface = ["llama-index-embeddings-huggingface"] embeddings-openai = ["llama-index-embeddings-openai"] embeddings-sagemaker = ["boto3"] embeddings-azopenai = ["llama-index-embeddings-azure-openai"] vector-stores-qdrant = ["llama-index-vector-stores-qdrant"] vector-stores-chroma = ["llama-index-vector-stores-chroma"] vector-stores-postgres = ["llama-index-vector-stores-postgres"] storage-nodestore-postgres = ["llama-index-storage-docstore-postgres", "llama-index-storage-index-store-postgres", "psycopg2-binary", "asyncpg"]

[tool.poetry.dev-dependencies] black = "^22.3" mypy = "^0.910" pre-commit = "^2.15.0" pytest = "^6.2.5" pytest-cov = "^3.0.0" ruff = "^0.2.2" pytest-asyncio = "^0.15.1" types-pyyaml = "^6.0.1"

[build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api"

Ragavendra-Vigneshwaran-R commented 3 months ago

@invisiblepancake Tried your configuration in my setup. But issue still there. Please let me know if there is any other solution:)

invisiblepancake commented 3 months ago

poetry install --extras "ui llms-llama-cpp embeddings-huggingface vector-stores-qdrant"

poetry run python scripts/setup

PGPT_PROFILES=local make run

? =)

edd: from gpt root dir ofc (bash from/to the right directory...heheh, got m so many times), and need to get pythons to babblefish? ull figgure by then =)

invisiblepancake commented 3 months ago

when i make run: inky@pinky:~$ cd privateGPT/ winky@dinky:~/privateGPT$ PGPT_PROFILES=ollama make run

" poetry run python -m private_gpt 21:12:00.501 [INFO ] private_gpt.settings.settings_loader - Starting application with profiles=['default', 'ollama'] 21:12:07.568 [INFO ] private_gpt.components.llm.llm_component - Initializing the LLM in mode=ollama 21:12:08.854 [INFO ] private_gpt.components.embedding.embedding_component - Initializing the embedding model in mode=ollama 21:12:08.859 [INFO ] llama_index.core.indices.loading - Loading all indices. 21:12:09.128 [INFO ] private_gpt.ui.ui - Mounting the gradio UI, at path=/ 21:12:09.205 [INFO ] uvicorn.error - Started server process [464000] 21:12:09.205 [INFO ] uvicorn.error - Waiting for application startup. 21:12:09.205 [INFO ] uvicorn.error - Application startup complete. 21:12:09.206 [INFO ] uvicorn.error - Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit) ^C21:12:15.523 [INFO ] uvicorn.error - Shutting down 21:12:15.623 [INFO ] uvicorn.error - Waiting for application shutdown. 21:12:15.624 [INFO ] uvicorn.error - Application shutdown complete. 21:12:15.624 [INFO ] uvicorn.error - Finished server process [464000] Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/home/pinky/privateGPT/private_gpt/main.py", line 11, in uvicorn.run(app, host="0.0.0.0", port=settings().server.port, log_config=None) File "/home/pinky/privateGPT/.venv/lib/python3.10/site-packages/uvicorn/main.py", line 575, in run server.run() File "/home/pinky/privateGPT/.venv/lib/python3.10/site-packages/uvicorn/server.py", line 65, in run return asyncio.run(self.serve(sockets=sockets)) File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "uvloop/loop.pyx", line 1511, in uvloop.loop.Loop.run_until_complete File "uvloop/loop.pyx", line 1504, in uvloop.loop.Loop.run_until_complete File "uvloop/loop.pyx", line 1377, in uvloop.loop.Loop.run_forever File "uvloop/loop.pyx", line 555, in uvloop.loop.Loop._run File "uvloop/loop.pyx", line 474, in uvloop.loop.Loop._on_idle File "uvloop/cbhandles.pyx", line 83, in uvloop.loop.Handle._run File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run File "/home/pinky/privateGPT/.venv/lib/python3.10/site-packages/uvicorn/server.py", line 68, in serve with self.capture_signals(): File "/usr/lib/python3.10/contextlib.py", line 142, in exit next(self.gen) File "/home/pinky/privateGPT/.venv/lib/python3.10/site-packages/uvicorn/server.py", line 328, in capture_signals signal.raise_signal(captured_signal) KeyboardInterrupt make: *** [Makefile:36: run] Interrupt

"

its so nice to have for your os =)