search

guidance-ai / guidance

A guidance language for controlling large language models.
MIT License
19.1k stars 1.04k forks source link

Ollama support #599

Open lestan opened 9 months ago

lestan commented 9 months ago

I use Ollama as my inference server for local LLMs. Ollama is supported by many LLM frameworks, but not Guidance.

Would love to see a direct integration with Ollama via the models package.

I'm aware that LiteLLM support is available and can be used to proxy Ollama, but that adds overhead and makes the solution more complex.

Supporting Ollama would immediately enable support for all the models Ollama makes available: model library

eliranwong commented 7 months ago

May I ask if any updates?

0xdevalias commented 5 months ago

See also: