-
**LocalAI version:**
v2.4.1
**Environment, CPU architecture, OS, and Version:**
MBP 14 M1 PRO
**Describe the bug**
Not working make build and make BUILD_TYPE=metal build
**To Reproduce…
-
I am trying to make working with GPU Tinyllama with:
```bash
./TinyLlama-1.1B-Chat-v1.0.F32.llamafile -ngl 9999
```
But it seem not possible to allocate 66.50 MB of memory on my card, even if I j…
-
Information about HF ZeroGPU Spaces can be found here: https://huggingface.co/zero-gpu-explorers
The environment and code for this issue is kept fully within this Hugging Face space, specifically t…
mgoin updated
5 months ago
-
I'd like to benchmark the optimized performance of LLAMA2 model with BigDL acceleration on SPR machine.
I followed the README in [python/llm/example/CPU/Native-Models](https://github.com/intel-anal…
-
- System environment:
bigdl-llm version: 2.4.0b20230921
OS: windows
CPU: Intel Core 13950HX/64GB memory
I'd like to run llama2 on CPU
- Procedure to reproduce the issue:
run …
-
### What is the issue?
[root@ubuntu]$ ollama run magnum-v2:12b-fp16
Error: exception error loading model hyperparameters: iinvalid n_rot: 160, expected 128
[root@ubuntu]$ ollama run magnum-v2.5:1…
-
I presume there is a minimum CPU requirement like needing AVX2, AVX-512, FP16C or something?
Could you document the minimum instruction set and extensions required.
root@1d1c4289f303:/llm-api# p…
dbzoo updated
9 months ago
-
Oogabooga text-generation-web-ui engine used for inference (prompts directly input into the oogabooga ui produce normal results but chat-ui is doing something weird as below), Mongodb setup
_**Prom…
-
I'm using docker's latest-aio-gpu-nvidia-cuda-12. I'm using multiple GPUs.
I would like to change the settings of llama-cpp in detail, but which file should I change?
I am modifying aio/gpu-8g/t…
-
Hi,
I'm trying to run a rayllm as the tutorial in README.
But now my serving seemed to stuck at replica. It looked like this:
![image](https://github.com/ray-project/ray-llm/assets/101038773/…