When I tried to run the Python scripts I always had the following error (just the beginning...):
gguf_init_from_file: invalid magic characters tjggÄ.
error loading model: llama_model_loader: failed to load model from E:/llama2-test/llama-2-7b-chat.ggmlv3.q8_0.bin
..........
Regarding https://swharden.com/blog/2023-07-29-ai-chat-locally-with-python/
Hi, thanks for your post, it's very interesting.
When I tried to run the Python scripts I always had the following error (just the beginning...): gguf_init_from_file: invalid magic characters tjggÄ. error loading model: llama_model_loader: failed to load model from E:/llama2-test/llama-2-7b-chat.ggmlv3.q8_0.bin ..........
After some googling I found this advice: You are trying to run .bin (ggml) model. Right now the correct format is gguf. You can download correct models at https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF
I downloaded the other model in gguf format from: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/blob/main/llama-2-7b-chat.Q8_0.gguf
and ran your Python scripts. Now it works!
Bye Cristian