-
### What happened?
When using llama.cpp models (e.g., granite-code and llama3) with Nvidia GPU acceleration (nvidia/cuda:12.6.1-devel-ubi9 and RTX 3080 10GB VRAM), the models occasionally return nons…
-
### What happened?
Was running Mistral Large 2 with partial offload with AMD 5600X + RTX 3090.
Provided the same ~28k prompt to each, llama.cpp produced output that was coherent and similar to non q…
-
### Cortex version
cortex-1.0.0-rc1-windows-amd64-local-installer
### Describe the Bug
Run all models by imported，return “ Model failed to load with status code: 500”
### Steps to Reproduce
1.cor…
-
### What is the issue?
When using the llm benchmark with ollama https://github.com/MinhNgyuen/llm-benchmark , I get around 80 t/s with gemma 2 2b. When asking the same questions to llama.cpp in conve…
-
I've been working on securing the user input, escaping invalid characters, however I've encountered a few prompts which cause the llama-cli to abruptly halt:
```
.\llama-cli.exe --model "..\..\..\mod…
-
### What happened?
Greetings,
I am not sure if this is a bug or unimplemented feature. I am using terminal for interactive conversations. I just realise that once you typed too much and you swappe…
-
### Contact Details
_No response_
### What happened?
I just downloaded [Meta-Llama-3.1-8B-Instruct.Q5_K_M.llamafile](https://huggingface.co/Mozilla/Meta-Llama-3.1-8B-Instruct-llamafile/blob/m…
-
I installed pyNGL (version 1.6.1) onto my Mac (OS 10.14.6) using the following commands:
conda create --name pyn_env --channel conda-forge pynio pyngl
source activate pyn_env
I then tested pyNG…
-
Can someone make the bot change status. Like i mean that the bot is changing between multiple statuses. Like this "website" "50 servers" "100 users". The bot changes between those.
Make the bot so…
-
### What happened?
I have managed to run the rpc server on 2 different machines running ubuntu (with different IPs) with the following commands:
1st machine: _bin/rpc-server -H MY_PUPLIC_IP -p 500…