-
I am using the base code as given in HF
from ctransformers import AutoModelForCausalLM
llm = AutoModelForCausalLM.from_pretrained('marella/gpt-2-ggml')
print(llm('AI is going to'))
While this…
-
Is there any chance of a future integration with CTransformers or something similar to allow for guided generation using quantized models on CPU? If I were to try and hack away at this, what would be …
-
Tracker to add support to ctransformers https://github.com/marella/ctransformers
https://github.com/jllllll/ctransformers-cuBLAS-wheels
-
I am using the same app.py, but there seems to be some issue with this line
`llm = CTransformers(
model_type='llama',
model='models/llama-2-7b.ggmlv3.q8_0.bin',
…
smd29 updated
6 months ago
-
GGML_ASSERT: D:\a\ctransformers\ctransformers\models\ggml/llama.cpp:453: data
I get this error sometimes when loading a model. At first, I thought it was a corrupted model, and I redownloaded it wh…
-
Chào bác, cám ơn bác đã đóng góp model, em có thử sử dụng model với RAG để sử dụng cho công việc, thì bị lỗi này:
RuntimeError: Failed to create LLM 'mpt' from 'models/PhoGPT-4B-Chat-Q4_K_M.gguf'.
…
-
You can modify the location where the Hugging Face model should be stored (when using the `transformers` library) by setting the environment variable `TRANSFORMERS_CACHE`. The default location is the …
-
im trying to have ctransformers use gpu but it won't work.
my chatdocs.yml:
```yml
ctransformers:
model: TheBloke/Wizard-Vicuna-7B-Uncensored-GGML
model_file: Wizard-Vicuna-7B-Uncensored…
-
### System Info
Python 3.11.2
Debian 12
ctransformers 0.2.27
transformers 4.42.4
### Who can help?
@ArthurZucker
### Information
- [X] The official example sc…
-
Hi, i can run below code last week without problem but i got below error since some days ago (after upgrade ctransformers lib).
I am unable to run ctransformers with all local LLM model now.
Can any…