search

keldenl / gpt-llama.cpp

A llama.cpp drop-in replacement for OpenAI's GPT endpoints, allowing GPT-powered apps to run off local llama.cpp models instead of OpenAI.
MIT License
594 stars 66 forks source link

stuck #41

Open C0deXG opened 1 year ago

C0deXG commented 1 year ago

am fallowing the intructions to install but i hade to chnage the numpy version to 1.19.0 to work and also i install vicuna.bin to run my model since there is no bulided model with the repo so when i download the vicuna.bin from fastChat repo and create 7B folder in the models folder and i put the ggml-vocab.bin there i run this command from the intructions: ./main -m models/7B/ggml-vocab.bin -p "the sky is" i get this:

command: ./main -m models/7B/ggml-vocab.bin -p "the sky is"

error: main: build = 526 (e6a46b0) main: seed = 1683697939 llama.cpp: loading model from models/7B/ggml-vocab.bin error loading model: missing tok_embeddings.weight llama_init_from_file: failed to load model llama_init_from_gpt_params: error: failed to load model 'models/7B/ggml-vocab.bin' main: error: unable to load mode

can i skip this part and how can i go forward if there is not model bulided in with this repo?

plz help if you can also if i try to host the backend as a API how is that possible since am just using localhost::8080 as a backend endpoint

keldenl commented 1 year ago

you need an actual model – ggml-vocab.bin isn't a model. you need to download it online u can find plenty on hugging face, just make sure it's labelled ggml

C0deXG commented 1 year ago

you need an actual model – ggml-vocab.bin isn't a model. you need to download it online u can find plenty on hugging face, just make sure it's labelled ggml

am getting error npm: npm install ─╯

gpt-llama.cpp@0.2.4 postinstall npm run updateengines && cd InferenceEngine/embeddings/all-mpnet-base-v2 && python -m pip install -r requirements.txt

gpt-llama.cpp@0.2.4 updateengines git submodule foreach git pull

sh: python: command not found npm ERR! code 127 npm ERR! path /Users/khederyusuf/Desktop/llama.cpp/gpt-llama.cpp npm ERR! command failed npm ERR! command sh -c npm run updateengines && cd InferenceEngine/embeddings/all-mpnet-base-v2 && python -m pip install -r requirements.txt

npm ERR! A complete log of this run can be found in: npm ERR! /Users/khederyusuf/.npm/_logs/2023-05-12T10_54_16_481Z-debug-0.log