Error when running run_localGPT.py with the --device_type cpu flag

[noahsnowc]

Actions taken:

Ran the command python run_localGPT.py --device_type cpu

Ingest.py --device_type cpu was ran before this with no issues.

Expected result:

For the "> Enter a query:" prompt to appear in terminal

Actual Result:

OSError: Unable to load weights from pytorch checkpoint file for 'C:\Users\/.cache\huggingface\hub\models--TheBloke--vicuna-7B-1.1-HF\snapshots\c3efe0b1dd78716c6bfc288a997026354bce441a\pytorch_model-00001-of-00002.bin' at 'C:\Users\/.cache\huggingface\hub\models--TheBloke--vicuna-7B-1.1-HF\snapshots\c3efe0b1dd78716c6bfc288a997026354bce441a\pytorch_model-00001-of-00002.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

Additional info:

Adding from_tf=True, after this "model = LlamaForCausalLM.from_pretrained(model_path, (from_tf=True added here)"

gives me the error: OSError: TheBloke/vicuna-7B-1.1-HF does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.

I have re downloaded the model multiple times now.

[AnnyD-9]

python run_localGPT.py this command is showing syntax error when run on google colab

[GaiaGrower]

I, too, ran: python run_localGPT.py --device_type cpu in an Anaconda shell on a Windows 10 machine. The error I got is: 2023-09-04 20:44:58,753 - INFO - run_localGPT.py:50 - Using Llamacpp for GGML quantized models llama.cpp: loading model from C:\Users\Owner.cache\huggingface\hub\models--TheBloke--Llama-2-7B-Chat-GGML\snapshots\00109c56c85ca9015795ca06c272cbc65c7f1dbf\llama-2-7b-chat.ggmlv3.q4_0.bin error loading model: unknown (magic, version) combination: 67676a74, 00000003; is this really a GGML file? llama_init_from_file: failed to load model Traceback (most recent call last): File "D:\Projects\libraries\GPTs\localGPT\localGPT-main\run_localGPT.py", line 246, in main() File "C:\Users\Owner\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1130, in call return self.main(args, kwargs) File "C:\Users\Owner\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1055, in main rv = self.invoke(ctx) File "C:\Users\Owner\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "C:\Users\Owner\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 760, in invoke return __callback(args, kwargs) File "D:\Projects\libraries\GPTs\localGPT\localGPT-main\run_localGPT.py", line 209, in main llm = load_model(device_type, model_id=MODEL_ID, model_basename=MODEL_BASENAME) File "D:\Projects\libraries\GPTs\localGPT\localGPT-main\run_localGPT.py", line 63, in load_model return LlamaCpp(kwargs) File "pydantic\main.py", line 341, in pydantic.main.BaseModel.initpydantic.error_wrappers.ValidationError: 1 validation error for LlamaCpp root Could not load Llama model from path: C:\Users\Owner.cache\huggingface\hub\models--TheBloke--Llama-2-7B-Chat-GGML\snapshots\00109c56c85ca9015795ca06c272cbc65c7f1dbf\llama-2-7b-chat.ggmlv3.q4_0.bin. Received error (type=value_error)

The file exists: From my Windows 10 File Manager: C:\Users\Owner.cache\huggingface\hub\models--TheBloke--Llama-2-7B-Chat-GGML\snapshots\00109c56c85ca9015795ca06c272cbc65c7f1dbf\llama-2-7b-chat.ggmlv3.q4_0.bin

Please advise. Thank you.