-
Hi everyone!
I've been using `llama-rs` recently and I especially love the caching session feature. I've been building an API that allows users to maintain multiple conversation threads and switch…
-
See https://huggingface.co/Bradarr/gpt4-x-alpaca-13b-native-ggml-model-q4_0
```
cargo run --release -- -m ./gpt4-x-alpaca-13b-native-ggml-model-q4_0.bin -p "How do you do ?"
```
```
thread 'm…
-
Following the same steps works for 7B and 13B model, with the 30B parameters I get
```
thread 'main' panicked at 'Could not load model: Tensor tok_embeddings.weight has the wrong size in model fil…
-
AutoConverter work and produces fp16weights (13GB for llama 7b) these can be loaded as wel…
-
Function for measuring context windows
1. A tokenisation algo
2. number of tokens to split to
3. Returns a Vec where each string can fit the window.
-
First, for the `revokePolicy` method that loops over all roles:
1. Check if we can revoke a policy if `numRoles` is 255 AND the user holds 0 roles
2. Check if we can revoke a policy if `numRoles` is…
-
Hi, I am trying to figure out the problem, help will be great.
Failing on windows 11 at cargo build --release
Error message:
`Caused by:
process didn't exit successfully: `C:\Users\touhidu…
-
Hi,
I think there is possibly data race in the function `ggml_compute_forward_mul_mat`.
From my understanding, the problem is that we are mutating c matrix, `cp`. But there is a possibility of dif…
-
### System Info
The error as specified in the title occurs with various nvidia/cuda configurations 530/12.1 530/11.8 470/11.4 etc. and with various version of pythia (sft-1 and sft-4) and starcoder. …
-
I fixed `main.rs` to refer to `&args.model_path`, but now I get a new error:
```
Could not load model: invalid utf-8 sequence of 1 bytes from index 0
```
I created these models using the tools…