Closed jcl2023 closed 1 year ago
Since it passes checks on Ubuntu. This might be caused by an package issue. Did you install LlamaCpp with GPU support?
Try pip install --upgrade cachecontrol
quick fix; remove n_gpu_layers inside startLLM.py here; here
Also your model ggml-vicuna-7b-4bit-**rev1**.bin
looks interesting. I'm sleep-writing atm so I'll hit you up in a few hours.
Work fine by Remving quick fix; remove n_gpu_layers inside startLLM.py. Wait for a proper fix.
Thanks,
Work fine by Remving quick fix; remove n_gpu_layers inside startLLM.py.
You forgot to run poetry install
after updating. You have outdated packages.
When I run "poetry install", it has the following errors:
Installing dependencies from lock file
Package operations: 29 installs, 10 updates, 0 removals
• Updating numpy (1.24.3 -> 1.23.5): Failed
KeyringLocked
Failed to unlock the collection!
at ~/anaconda3/envs/privateGPT/lib/python3.11/site-packages/keyring/backends/SecretService.py:67 in get_preferred_collection 63│ raise InitError("Failed to create the collection: %s." % e) 64│ if collection.is_locked(): 65│ collection.unlock() 66│ if collection.is_locked(): # User dismissed the prompt → 67│ raise KeyringLocked("Failed to unlock the collection!") 68│ return collection 69│ 70│ def unlock(self, item): 71│ if hasattr(item, 'unlock'):
Looks like a poetry issue unrelated to our repo. Try https://stackoverflow.com/questions/74438817/poetry-failed-to-unlock-the-collection
poetry works after "export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring"
.env
Generic
TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=false
Ingestion
PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 INGEST_N_THREADS=3
Generation
MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp
MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin
MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vicuna-7b-4bit-rev1.bin MODEL_TEMP=0.8 MODEL_N_CTX=1024 # Max total size of prompt+answer MODEL_MAX_TOKENS=256 # Max size of answer MODEL_STOP=[STOP] CHAIN_TYPE=betterstuff N_RETRIEVE_DOCUMENTS=100 # How many documents to retrieve from the db N_FORWARD_DOCUMENTS=100 # How many documents to forward to the LLM, chosen among those retrieved N_GPU_LAYERS=4
Python version
Python 3.11.3
System
Ubuntu 18.04.4 LTS
CASALIOY version
latest master
Information
Related Components
Reproduction
Errors:
54 │ │ │ case "LlamaCpp": │ │ 55 │ │ │ │ from langchain.llms import LlamaCpp │ │ 56 │ │ │ │ │ │ ❱ 57 │ │ │ │ llm = LlamaCpp( │ │ 58 │ │ │ │ │ model_path=model_path, │ │ 59 │ │ │ │ │ n_ctx=n_ctx, │ │ 60 │ │ │ │ │ temperature=model_temp, │ │ │ │ in pydantic.main.BaseModel.init:341
ValidationError: 1 validation error for LlamaCpp n_gpu_layers extra fields not permitted (type=value_error.extra)
Expected behavior
Should not have an error