Closed Fulgurance closed 1 month ago
You should download a model firstly, mate.
If you check your models dir, probably you cannot find llama-7b.
Sorry I am starting with llama. So I clone that model for example: https://huggingface.co/THUDM/glm-4-9b
But I don't see any .gguf file. I guess I have to generate it ? How I am suppose to do that ?
For this kind of model, you need to convert it to gguf, use convert-hf-to-gguf.py
to do it, you can find details in documentation.
However, if you just want a quick start, have a try of "quantised-ready" model, like this: https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF
So I tried to run your model. But I get an error again.
zohran@alienware-m17-r3 ~/Downloads/llama.cpp-b3400 $ ./llama-cli -m /home/zohran/Downloads/Llama-2-13B-chat-GGUF/llama-2-13b-chat.Q8_0.gguf -p "How are you?"
Log start
main: build = 0 (unknown)
main: built with cc (Gentoo Hardened 14.1.1_p20240622 p2) 14.1.1 20240622 for x86_64-pc-linux-gnu
main: seed = 1721145009
gguf_init_from_file: invalid magic characters 'vers'
llama_model_load: error loading model: llama_model_loader: failed to load model from /home/zohran/Downloads/Llama-2-13B-chat-GGUF/llama-2-13b-chat.Q8_0.gguf
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model '/home/zohran/Downloads/Llama-2-13B-chat-GGUF/llama-2-13b-chat.Q8_0.gguf'
main: error: unable to load model
On huggingface, there is a demo code for llama.cpp (at the top-right corner "Use this model" button). Probably have a try:
./llama-cli --hf-repo "TheBloke/Llama-2-13B-chat-GGUF" -m llama-2-13b-chat.Q2_K.gguf -p "How are you?" -n 128
Okay I understood how to download now.
So I have a question. Which model do you recommend me ? Because I would like to integrant an IA assistant inside the linux distribution I am making, and I would like to teach the assistant how to manage the system with my tools. The one you gave me is good you think for that ? Basically I would like to this assistant to be able to run command.
I would like as well when the AI answer to stop talking too much xD. How can I allow the AI to run some bash command in my system ?
Because the terminal is always showing extra text, I want to avoid that (with im_end and tips):
> Hello
Hi there! How can I help you today?
<|im_end|>
In this example, the
>
Okay I understood how to download now.
So I have a question. Which model do you recommend me ? Because I would like to integrant an IA assistant inside the linux distribution I am making, and I would like to teach the assistant how to manage the system with my tools. The one you gave me is good you think for that ? Basically I would like to this assistant to be able to run command.
I would like as well when the AI answer to stop talking too much xD. How can I allow the AI to run some bash command in my system ?
TBH, I don't think you're going to get a good answer to that question here. You're clearly new to this stuff and have a lot of homework to do. What you want to do is extremely complicated and probably well out of reach for your skill level. My advice is to start doing a lot of research and attempt far easier projects first. You're asking how to design a car when you don't know how to drive. Also, this is not the appropriate place for these questions. I figured this comment would be more helpful to you than silence. Good luck
I understood how to do now no worries 👍 I asked somewhere else with people more able to explain. It's not that much complicate...
I understood how to do now no worries 👍 I asked somewhere else with people more able to explain. It's not that much complicate...
Let me know when you're done so I can check it out!
What happened?
Hi guys. I have got a problem after I compile Llama on my machine. It built properly, but when I try to run it, it is looking for a file don't even exist (a model).
Is it normal ?
Name and Version
version: 0 (unknown) built with cc (Gentoo Hardened 14.1.1_p20240622 p2) 14.1.1 20240622 for x86_64-pc-linux-gnu
What operating system are you seeing the problem on?
Linux
Relevant log output