[completions and embeddings] Add support for Ollama (local LLMs)

LangStream / langstream

LangStream. Event-Driven Developer Platform for Building and Running LLM AI Apps. Powered by Kubernetes and Kafka.

https://langstream.ai

Apache License 2.0

379 stars 28 forks source link

Closed eolivelli closed 7 months ago

eolivelli commented 7 months ago

Summary:

add support for https://ollama.ai/
with this patch you can now connect to local/custom LLMs via the Ollama REST API
add a sample application that uses Ollama both for Embeddings and completions

Configuration for Ollama

resources:
    - type: "ollama-configuration"
      name: "ollama"
      configuration:
        url: "${secrets.ollama.url}"