-
## Describe the bug
Download https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.2/mistralrs-server-aarch64-apple-darwin.tar.xz
Use a tool like [asitop](https://github.com/tlkh/asito…
-
**Describe the bug**
llama-server exited with status code -1
**Information about your version**
Unable to get version as it will not start. Docker image used:
```
REPOSITORY …
-
**Describe the bug**
(*This might be me doing something wrong, and not a bug!*)
I can't seem to get Tabby to run using my GPU (Radeon 6600 XT). Not with ROC, (which I believe is unsupported for …
-
**Describe the bug**
Tabby ignores the --device setting and always runs on CPU
**Information about your version**
```
ldd target/debug/tabby
linux-vdso.so.1 (0x00007fff91979000)
…
-
I meet coredump when decoding with multi-thread. It cored in rust function `tokenizers_decode`,rust/src/lib.rs:199. here is the core backtrack.
why does it do not support multi-thread? I think dec…
-
### Summary
I'm trying to run llama-api-server with Llama-3-8B in wasm-sandboxer of Kuasar! But got some errors.
### Current State
_No response_
### Expected State
_No response_
### …
-
**Describe the bug**
On the latest RC (0.17.0-rc3) I see the following logs when starting tabby
```
Starting...2024-09-06T21:51:10.063326Z WARN llama_cpp_server::supervisor: crates/llama-cpp-server/…
-
gen_vulkan_shaders failed at
```
Command::new(vulkan_shaders_gen_bin)
.args([
"--glslc".as_ref(), "glslc".as_ref(),
"--input-dir".as_ref(), vulkan_shaders_src…
-
### System Info
```shell
AWS EC2 instance: trn1.32xlarge
OS: Ubuntu 22.04.4 LTS
Platform:
- Platform: Linux-6.5.0-1023-aws-x86_64-with-glibc2.35
- Python version: 3.10.12
Python packages:
…
-
Hey
I have a suggestion that I believe could enhance its flexibility and ease of use:
Would it be possible to implement example [floneum/models/xxx/src/source.rs](https://github.com/floneum/flon…