search

PromtEngineer / localGPT

Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.
Apache License 2.0
20.13k stars 2.25k forks source link

Unable to build wheel for llama-cpp-python #433

Open kikoferrer opened 1 year ago

kikoferrer commented 1 year ago

I am trying to install this on my Linux Mint 21 laptop and got this error below. I used the command CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install -r requirements.txt.

The log was alot longer but I only copied the headings. What was missing? It can't build wheel for llama-cpp-python. I use venv as well. Thanks

Building wheels for collected packages: llama-cpp-python, auto-gptq, sentence-transformers, docx2txt, htbuilder, st-annotated-text Building wheel for llama-cpp-python (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully. │ exit code: 1

Octavian11 commented 1 year ago

I had this issue and took me 2 days to figure it out. For me, it had to do with GCC compiler. Try to install development tools with commands below.

yum groupinstall 'Development Tools' yum install python3-devel

andyra42 commented 1 year ago

I got similiar issue to resolve the issue end up installing nvidia tool kit on ubuntu ec2 server apt install nvidia-cuda-toolkit

kikoferrer commented 1 year ago

I had this issue and took me 2 days to figure it out. For me, it had to do with GCC compiler. Try to install development tools with commands below.

yum groupinstall 'Development Tools' yum install python3-devel

How to do that with apt? Sorry I am not using Arch.

kikoferrer commented 1 year ago

I got similiar issue to resolve the issue end up installing nvidia tool kit on ubuntu ec2 server apt install nvidia-cuda-toolkit

I am having error installing cuda toolkit as I have 11.8 cuda installed. If I do try to install cuda toolkit means my other softwares will not work anymore......

andyra42 commented 1 year ago

i have installed the Latest Version of CUDA from the Official NVIDIA Package Repository followed the below link for instructions https://linuxhint.com/install-cuda-ubuntu-2004/ It will not break any other software's

kikoferrer commented 1 year ago

i have installed the Latest Version of CUDA from the Official NVIDIA Package Repository followed the below link for instructions https://linuxhint.com/install-cuda-ubuntu-2004/ It will not break any other software's

Thank you this worked! The problem was the $PATH and it solved my toolkit issue. Now I have to retry if this will install properly this time. Thanks

kikoferrer commented 1 year ago

It's working now. I was able to ingest a small pdf document. now it is running and asking me a query. I asked a query and it returned

ggml_new_object: not enough space in the context's memory pool (needed 18896624, available 10650320) Segmentation fault (core dumped)

What went wrong here? is it the model context size too small? My VRAM was not even fully used.

andyra42 commented 1 year ago

change the context size to 4096 for llama2. and disable the history by default.