-
## Describe the bug
Have a look :-)
https://github.com/user-attachments/assets/321dbb21-2403-4330-9ce1-091902298888
## Latest commit or version
0.22
MBP M3 Max
-
```
error occurred: Command ZERO_AR_DATE="1" "ar" "cq" "/home/tc-wolf/rust-llama.cpp/target/release/build/llama_cpp_rs-75252caa56296e09/out/libbinding.a" "/home/tc-wolf/rust-llama.cpp/target/releas…
-
On adding llama_cpp-rs to my Cargo.toml, llama.cpp seems to be locked to an older version. I'm trying to use Phi-3 128k in a project and I'm unable to because the [PR that was merged into llama.cpp](h…
-
how convert microsoft/Phi-3.5-MoE-instruct to gguf?
-
Hi there,
First thank you for unsloth, it's great!
I've finetuned a llama-3-8b-Instruct-bnb-4bit and pushed it to hf hub. When I try to deploy it using [hf Inference Endpoints](https://huggingfa…
-
With the recent advent of large models (take Llama 3.1 405b, for example!), distributed inference support is a must! We currently support naive device mapping, which works by allowing a combination of…
-
Hello, I'm not sure if multi GPU is supported yet. I didn't find parameters for tensor parallel, and the "num_device_layers" parameter seems not work. Please let me know if it supports or has plans to…
-
**Describe the bug**
start tabby serve on docker desktop, no logs,and cannot open http://localhost:9090/
**Information about your version**
Please provide output of `tabby --version`
tabby 0.12.…
-
Hi, really appreciate the work on this project (the llama-cpp-rs crates are not great).
I would like to load a text embedding model such as: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-G…
-
I'm trying to deploy Llama3 8b on GKE using optimum but running into some troubles.
Following instructions here: https://github.com/huggingface/optimum-tpu/tree/main/text-generation-inference. I bu…