-
Using the command `$ CC="/opt/rocm/llvm/bin/clang" CXX="/opt/rocm/llvm/bin/clang++" CT_HIPBLAS=1 pip install ctransformers --no-binary ctransformers` I am unable to compile ctransformers for ROCM. I'v…
-
After follow the guide at last step https://postgresml.org/docs/guides/unified-rag#unified-retrieval-+-reranking-+-text-generation
I got this error after run the query (via Docker)
I think is crash …
-
I'm using `cTransformers` with `streamlit` to provide a UI to an LLM. After running the model once on my machine and closing the application and trying to start up again, I get repeated segmentation f…
-
### Feature request
https://huggingface.co/docs/transformers/main/en/gguf
in above documentation it shows that it loads the gguf model and provided the simple example
from transformers import Aut…
-
When i run ctransformers[cuda], i get the error:
CUDA error 35 at /home/runner/work/ctransformers/ctransformers/models/ggml/ggml-cuda.cu:4236: CUDA driver version is insufficient for CUDA runtime ver…
-
is there instructs for that - most of us AI folk have good gpus - seems silly not to use them.
-
Should be pretty doable. This model would run nicely on a T4 or equivalent hardware: https://huggingface.co/TheBloke/Llama-2-13B-GGUF
-
Hello, I'm trying to use ctransformers as below:
```
from ctransformers import AutoModelForCausalLM
# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is …
-
I have installed ctransformers using -
`pip install ctransformers[cuda]`
I am trying following piece of code -
```
from langchain.llms import CTransformers
config = {'max_new_tokens': 512, '…
-
I'd love to use ctransformers with the [outlines](https://github.com/normal-computing/outlines) library for constrained generation. I opened this [issue ](https://github.com/normal-computing/outlines/…