Can I use Ollama model - Githubissues

b4rtaz / distributed-llama

Tensor parallelism is all you need. Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.

MIT License

1.03k stars 69 forks source link

Can I use Ollama model #16

Closed liyimeng closed 3 months ago

liyimeng commented 3 months ago

Sorry if asking a stupid question. I am newbie llm user, have been using Ollama and love its ease of use. However, I also have limit on the hardware. distributed-llama seem a promising solution for me. But I don't know how to use those model provided by Ollama. o is it feasible at all?

b4rtaz commented 3 months ago

Hello @liyimeng! Unfortunately Distributed Llama doesn't support Ollama models.