-
While loading the model, the program exits with the following error:
```
whisper_init_from_file_with_params_no_state: loading model from '/Users/bochao/models/whisper/large-v3/ggml-large-v3-q5_0.bin…
-
I've been working on a a native Rust GBNF parser that generates `WhisperGrammarElementType` arrays as needed for whisper-rs. It's not particularly complicated - about 40 lines of [pest](https://docs.r…
jcsoo updated
4 months ago
-
it appears that the n_batch in sessions remain the default value of 512. I have had some large inputs of over 800 tokens testing Hermes mistral v0.2 and Dolphin Phi 2 and both had the same error of `n…
-
**Describe the bug**
Models seam to produce garbled output on very long prompts.
If i use the following script:
```python
import openai
from transformers import AutoTokenizer
if __name__ =…
-
```rust
use anyhow::{bail, Result};
use candle_core::utils::{cuda_is_available, metal_is_available};
use candle_core::{DType, Device, Tensor};
use candle_nn::VarBuilder;
use candle_transformers::…
-
### Bug Description
Got `TypeError: 'NoneType' object is not iterable` when I run `index = VectorStoreIndex.from_documents(documents)`
### Version
Latest version
### Steps to Reproduce
…
-
./bin/gaianet init
[+] Checking the config.json file ...
[+] Downloading Meta-Llama-3-8B-Instruct-Q5_K_M.gguf ...
* Using the cached Meta-Llama-3-8B-Instruct-Q5_K_M.gguf in /Users/XXX/gaianet…
-
Hi Author, I tried to implement llm-rs in my jupyter. After i installed RUST the error still occured :
thread '' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidMagic { path: "mode…
-
Yesterday running quantized models with CUDA was merged into candle:
https://github.com/huggingface/candle/pull/1754
I haven't tested it yet, but right now it looks very unstable as its in a ver…
-
https://github.com/rustformers/llama-rs/issues/4 mentions doing benchmarking. We should add some microbenchmarks to show the words per second v.s. llama.cpp. In fact, we should use a similar benchmark…