Does ctransformers support ollama models? - Githubissues

marella / ctransformers

Python bindings for the Transformer models implemented in C/C++ using GGML library.

MIT License

1.76k stars 137 forks source link

Does ctransformers support ollama models? #204

Open PriyaranjanMarathe opened 4 months ago

PriyaranjanMarathe commented 4 months ago

Does ctransformers support ollama models?

How do I specify the model in this code below?

llm = CTransformers(model="***where is the model file for a ollama model?", model_type="llama", max_new_tokens=512, temperature=0.1)

PriyaranjanMarathe commented 4 months ago

https://github.com/ollama/ollama/issues/2652