-
It is mentioned on README that candle supports multi GPU inference, using NCCL under the hood. How can this be implemented ? I wonder if there is any available example to look at..
Also, I know PyT…
-
I've been testing running various finetuned versions of supported models on GKE. However, it gets stuck on ` Using the Hugging Face API to retrieve tokenizer config`
This are the full logs
```…
-
```
➜ ~ gaianet start
[+] Checking the config.json file ...
You already have a private key.
[+] Starting Qdrant instance ...
Qdrant instance started with pid: 5538
[+] Starting Llam…
-
Can't utilize GPU on Mac with
```
llama_cpp_rs = { git = "https://github.com/mdrokz/rust-llama.cpp", version = "0.3.0", features = [
"metal",
] }
```
Code
```
use llama_cpp_rs::{
opti…
-
**Steps to Reproduce:**
1. Run `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash`
2. Run `source /root/.bashrc`
3. Run `/root/gaianet/bin/gaianet i…
-
Hi there,
i am trying to build simple chat example with the llma_cpp-rs crate, following is my code
```
use std::io;
use std::io::Write;
use llama_cpp::standard_sampler::StandardSampler;
us…
-
**Describe the bug**
I followed the guide in this doc: https://tabby.tabbyml.com/docs/quick-start/installation/apple/
after i installed tabby use brew, and when i run below command:
tabby serve --d…
-
### Prerequisites
- [X] I am running the latest code. Mention the version if possible as well.
- [X] I carefully followed the [README.md](https://github.com/ggerganov/llama.cpp/blob/master/README.md)…
-
## use docker image [docker pull ghcr.io/ericlbuehler/mistral.rs:cuda-80-sha-9b898ee]
run script in wsl2
```docker run --gpus all -v E:/workspace/modelscope:/root/modelscope ghcr.io/ericlb…
-
When I try on my window PC, it works.
docker run --entrypoint /opt/tabby/bin/tabby-cpu -it -p 18080:8080 -v D:\BaiduNetdiskDownload\TabbyML\tabbyml-chatWizardCoder-3B\tabbyml-chatWizardCoder-3B\.tabb…