an error when running python ./casalioy/startLLM.py

[jcl2023]

.env

Generic

TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=false

Ingestion

PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 INGEST_N_THREADS=3

Generation

MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp

MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin

MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vicuna-7b-4bit-rev1.bin MODEL_TEMP=0.8 MODEL_N_CTX=1024 # Max total size of prompt+answer MODEL_MAX_TOKENS=256 # Max size of answer MODEL_STOP=[STOP] CHAIN_TYPE=betterstuff N_RETRIEVE_DOCUMENTS=100 # How many documents to retrieve from the db N_FORWARD_DOCUMENTS=100 # How many documents to forward to the LLM, chosen among those retrieved N_GPU_LAYERS=4

Python version

Python 3.11.3

System

Ubuntu 18.04.4 LTS

CASALIOY version

latest master

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Related Components

• [ ] Document ingestion
• [ ] GUI
• [ ] Prompt answering

Reproduction

Errors:

54 │ │ │ case "LlamaCpp": │ │ 55 │ │ │ │ from langchain.llms import LlamaCpp │ │ 56 │ │ │ │ │ │ ❱ 57 │ │ │ │ llm = LlamaCpp( │ │ 58 │ │ │ │ │ model_path=model_path, │ │ 59 │ │ │ │ │ n_ctx=n_ctx, │ │ 60 │ │ │ │ │ temperature=model_temp, │ │ │ │ in pydantic.main.BaseModel.init:341

ValidationError: 1 validation error for LlamaCpp n_gpu_layers extra fields not permitted (type=value_error.extra)

Expected behavior

Should not have an error

[su77ungr]

Since it passes checks on Ubuntu. This might be caused by an package issue. Did you install LlamaCpp with GPU support?

Try pip install --upgrade cachecontrol

quick fix; remove n_gpu_layers inside startLLM.py here; here

Also your model ggml-vicuna-7b-4bit-**rev1**.bin looks interesting. I'm sleep-writing atm so I'll hit you up in a few hours.

[jcl2023]

Work fine by Remving quick fix; remove n_gpu_layers inside startLLM.py. Wait for a proper fix.

Thanks,

[hippalectryon-0]

Work fine by Remving quick fix; remove n_gpu_layers inside startLLM.py.

You forgot to run poetry install after updating. You have outdated packages.

[jcl2023]

When I run "poetry install", it has the following errors:

Installing dependencies from lock file

Package operations: 29 installs, 10 updates, 0 removals

• Updating numpy (1.24.3 -> 1.23.5): Failed

KeyringLocked

Failed to unlock the collection!

at ~/anaconda3/envs/privateGPT/lib/python3.11/site-packages/keyring/backends/SecretService.py:67 in get_preferred_collection 63│ raise InitError("Failed to create the collection: %s." % e) 64│ if collection.is_locked(): 65│ collection.unlock() 66│ if collection.is_locked(): # User dismissed the prompt → 67│ raise KeyringLocked("Failed to unlock the collection!") 68│ return collection 69│ 70│ def unlock(self, item): 71│ if hasattr(item, 'unlock'):

[hippalectryon-0]

Looks like a poetry issue unrelated to our repo. Try https://stackoverflow.com/questions/74438817/poetry-failed-to-unlock-the-collection

[jcl2023]

poetry works after "export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring"