-
### What happened?
I compiled `llama-llava-cli.exe` with Vulkan Support, I followed this document https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md
I tried this commnad
`./llama-…
-
Overview:
We want to use pytest's parameterize decorator function to enable the testing of multiple models of a provider.
For example:
We automatically test the LLM's default.
```
@pytest.m…
-
### 🚀 The feature, motivation and pitch
We have a deployment of Llama3.1-8B-Instruct and Llama3.1-70B-Instruct models through vLLM hosted in our OnPremise GPU infra.
While testing different use-ca…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
If I have a PDF file which I have transformed into a local markdown file `.md` and the i…
-
### Prerequisites
- [X] I am running the latest code. Mention the version if possible as well.
- [X] I carefully followed the [README.md](https://github.com/ggerganov/llama.cpp/blob/master/README.md)…
-
Failed to load model: No LM Runtime found for format 'safetensors!
Model: Phi-3.5-vision-instruct-gguf
![image](https://github.com/user-attachments/assets/0f9b1a82-c260-45bb-a286-9118dcefb33c)
…
-
### What happened?
Follow the steps in [README-minicpmv2.5.md#usage](https://github.com/ggerganov/llama.cpp/blob/master/examples/llava/README-minicpmv2.5.md#usage) to convert `minicpm v2.5`. The conv…
-
I tried two gguf conversion on M2 ultra (metal) but no luck. I converted them myself and still the same error.
Here is the first model I tried:
https://huggingface.co/guinmoon/MobileVLM-1.7B-GGUF…
-
- [Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct)
- [Phi-3.5-MoE-instruct](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct)
- [Phi-3.5-vision-instruct](https://…
-
That model is insane for its size ....
https://huggingface.co/microsoft/Phi-3-vision-128k-instruct