-
### What happened?
I was experimenting with the llama.cpp project and llm inference in general. I made a basic chat application (similar to the main.cpp project from the examples) but much simpler. N…
-
使用llama.cpp/convert_hf_to_gguf.py模型格式转换失败，7B的这个多模态模型是不允许转换吗，转换指令如下：
python convert_hf_to_gguf.py /data/houshiwei/qwen2_model/Qwen/Qwen2-VL-7B-Instruct --outfile /data/houshiwei/gguf_model
报错信息如下：
…
-
**Is your feature request related to a problem? Please describe.**
It would be nice to intergrate https://llama-cpp-python.readthedocs.io/en/stable/#embeddings because of the speed of default `senten…
-
I got the following error when running model Imported from GGUF which is generated from the model fine-tuned with LoRA.
Error: llama runner process has terminated: GGML_ASSERT(src1t == GGML_TYPE_F…
-
Seems like an absolutely awesome project. I do a lot of domain expert LLM finetuning so this would be amazing to have in my work. What has to be done to get this into common inference engines like lcp…
-
Zamba2-7B-Instruct is a hybrid model composed of state-space ([Mamba2](https://github.com/state-spaces/mamba)) and transformer blocks.
HF: https://huggingface.co/Zyphra/Zamba2-7B-Instruct
llama.…
-
### Describe what should be investigated or refactored
The LeapfrogAI llama-cpp-python backend has not been updated in a long time, dependencies wise and code-wise. It lacks the necessary code and …
-
### Background Description
Ref: https://github.com/ggerganov/llama.cpp/pull/7553 , required for supporting future vision models (https://github.com/ggerganov/llama.cpp/issues/8010)
I initially pla…
-
# Prerequisites
pip install llama-cpp-python --verbose
# Environment and Context
```
$ python3 --version
Python 3.12.3
$ make --version
GNU Make 3.82
$ g++ --version
gcc (GCC) 11.2.0
```…
-
# Prerequisites
When I install via pip install llama-cpp-python, there will be an error. It will occur on versions 0.2.81 and 0.2.80, The version 0.2.79 can be successfully installed.
python 3.11…