-
Hi @marella
Hope all is OK with you. I'm getting various reports on my repos with issues with CTransformers, and I see it's not been updated for nearly 2 months now.
Is everything OK? Should I …
-
~~~~~~~~~~~~~~~~~~~~~~~
Any use of "gpu_layers" crashes it.
~~~~~~~~~~~~~~~~~~~~~~~
CUDA is working:
(ct) C:\Users\Jeremy\Documents>python
Python 3.11.6 (tags/v3.11.6:8b6ee5b, Oct 2 2023…
-
## Expected Behavior
I would like ChatWithDocs read from a .txt file and use this information in the answer.
## Current Behavior
- Retreiving personality settings
- Setting personality settings
…
-
Just grabbed a clone today. I was struggling with the latest GGML models. Then read this requires ctransformers. I changed the model type and now it works fine. Can we change the default in the github…
-
on windows 10 (AMD Ryzen 5 5600) , gets the Runtime error :Failed to create LLM 'replit' from ......\models\replit-v2-codeinstruct-3b.q4_1.bin during inference (python inference.py) . it says Model …
-
After running python app.py it throws it throws -**There is a 500 error. Please contact admin for support.** in localhost.
-
## Expected Behavior
Running ./setup.sh install all required python modules
## Current Behavior
Loading binding c_transformers install ON
Traceback (most recent call last):
File "/home/gpt4al…
-
Quantized versions are here:
https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF
Q4_K_M
https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/blob/main/mistral-7b-instruct-v…
-
`llm = CTransformers(model="llama-2-7b-chat.ggmlv3.q8_0.bin",
streaming=True,
callbacks=[StreamingStdOutCallbackHandler()],
…
-
What's the best practice to interface ctransformers API and expose it as REST API?
I looked at https://github.com/jquesnelle/transformers-openai-api and tried to change it to use ctransformer, but…