search

zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks
https://privategpt.dev
Apache License 2.0
53.51k stars 7.19k forks source link

Segmentation fault (core dumped) #1817

Closed anamariaUIC closed 1 month ago

anamariaUIC commented 5 months ago

Hello,

I installed private GPT via:

rm -rf .local .cache /scratch/network/$USER/privateGPT /scratch/network/$USER/tmp
mkdir /scratch/network/$USER/tmp
export TMPDIR=/scratch/network/$USER/tmp
module load Python/3.11.3-GCCcore-12.3.0
pip install --user poetry
module load git
module load CUDA
export PATH=$PATH:/home/$USER/.local/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/$USER/.local/lib
cd /scratch/network/$USER
git clone https://github.com/imartinez/privateGPT.git
cd privateGPT/
export PYTHONPATH=$PYTHONPATH:$PWD
poetry install --extras "ui llms-llama-cpp embeddings-huggingface vector-stores-qdrant"
poetry run python scripts/setup
CMAKE_ARGS='-DLLAMA_CUBLAS=on' poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
PGPT_PROFILES=local make run

When I query a 3 pages long .pdf file I am getting: Segmentation fault (core dumped) error. Please see in attach details.

How this can be resolved?

Screen Shot 2024-04-01 at 4 01 33 PM Screen Shot 2024-04-01 at 4 27 15 PM
beaconai commented 5 months ago

i think your GPU utilization increased

beaconai commented 5 months ago

i'm facing the problem "Initial token count exceed token limit" can anyone help me out, im upload a CVS file and im getting this error.

anamariaUIC commented 5 months ago

I only queried a tiny .txt file one question and attempted to ask a 3 page .pdf one question when I received this segmentation error…

Does privateGPT has the run on a GPU?

Sent from Outlook for iOShttps://aka.ms/o0ukef

From: beaconai @.> Sent: Monday, April 1, 2024 10:03:26 PM To: zylon-ai/private-gpt @.> Cc: Sokovic, Ana Marija @.>; Author @.> Subject: Re: [zylon-ai/private-gpt] Segmentation fault (core dumped) (Issue #1817)

i think your GPU utilization increased

— Reply to this email directly, view it on GitHubhttps://github.com/zylon-ai/private-gpt/issues/1817#issuecomment-2030988851, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUHBH5AUACV6HUGRZCXEV4LY3INX5AVCNFSM6AAAAABFSHNKG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQHE4DQOBVGE. You are receiving this because you authored the thread.Message ID: @.***>

boufaka commented 5 months ago

I solve the same issue (segementation fault) changing a file in version 0.5.0:

privateGPT/private_gpt/components/llm/llm_component.py

The setting "offload_kqv": True to "offload_kqv": False

I hope this helps