-
Make CTransformers compatible with π€ Transformers:
- [x] Add Transformers-compatible model
- [x] Add Transformers-compatible tokenizer
See:
- https://github.com/marella/ctransformers/issues/β¦
-
Hi,
When trying to run inference, I got the following error message:
```
Downloading (β¦)feaf43e4/config.json: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ¦
-
Hi there. I have installed cuda toolkit 11.7 using conda. Also gcc (tested multiple versions). Here with 9.4.0 I get this error:
-- Build files have been written to: /nobackup/sa226037/6273β¦
-
i tried mistral & llama7b from ctransofrmer & getting this issue,is there any way to add support for this?
how can we implement it with websocket?
```
streaming_llm = CTransformers(model='Tβ¦
-
I am using the Linux system with GPU and installed ctransformers-0.2.14 using pip. It installed all fine. But now when I try to run the
GGML model quantized by @TheBloke (TheBloke/upstage-llama-30b-β¦
-
Hi! It would fantastic if it supported Llama3. I hit a bug, and I'm unsure whether or not it is caused by quantization or by Ctransformers not being able to handle Llama3.
ERROR: byte not found inβ¦
-
Any ideas on why this code doesn't work? I was trying to use sliding windows, but it seems like Mistral doesn't support that. So then I tried truncating, and setting a higher token limit - but that doβ¦
-
What cuda drivers are needed?
I'm able to run llama-cpp-pyhon and llama.cpp just fine, but not this tool
Ubuntu 22
Cuda 11.7
driver is 515.43.04
-
I tried to run on windows and i get this error
```
ERROR: Traceback (most recent call last):
File "C:\streamlit-llama\venv\lib\site-packages\starlette\routing.py", line 677, in lifespan
aβ¦
-
Does ctransformers support Sliding Window Attention (SWA) ?