-
When i run ctransformers[cuda], i get the error:
CUDA error 35 at /home/runner/work/ctransformers/ctransformers/models/ggml/ggml-cuda.cu:4236: CUDA driver version is insufficient for CUDA runtime ver…
-
i tried mistral & llama7b from ctransofrmer & getting this issue,is there any way to add support for this?
how can we implement it with websocket?
```
streaming_llm = CTransformers(model='T…
-
I'm using `cTransformers` with `streamlit` to provide a UI to an LLM. After running the model once on my machine and closing the application and trying to start up again, I get repeated segmentation f…
-
Should be pretty doable. This model would run nicely on a T4 or equivalent hardware: https://huggingface.co/TheBloke/Llama-2-13B-GGUF
-
I'd love to use ctransformers with the [outlines](https://github.com/normal-computing/outlines) library for constrained generation. I opened this [issue ](https://github.com/normal-computing/outlines/…
-
Hi there. I have installed cuda toolkit 11.7 using conda. Also gcc (tested multiple versions). Here with 9.4.0 I get this error:
-- Build files have been written to: /nobackup/sa226037/6273…
-
Hello, I'm trying to use ctransformers as below:
```
from ctransformers import AutoModelForCausalLM
# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is …
-
I have installed ctransformers using -
`pip install ctransformers[cuda]`
I am trying following piece of code -
```
from langchain.llms import CTransformers
config = {'max_new_tokens': 512, '…
-
Chào bác, cám ơn bác đã đóng góp model, em có thử sử dụng model với RAG để sử dụng cho công việc, thì bị lỗi này:
RuntimeError: Failed to create LLM 'mpt' from 'models/PhoGPT-4B-Chat-Q4_K_M.gguf'.
…
-
Make CTransformers compatible with 🤗 Transformers:
- [x] Add Transformers-compatible model
- [x] Add Transformers-compatible tokenizer
See:
- https://github.com/marella/ctransformers/issues/…