search

lennartpollvogt / ollama-instructor

Python library for the instruction and reliable validation of structured outputs (JSON) of Large Language Models (LLMs) with Ollama and Pydantic. -> Deterministic work with LLMs.
MIT License
67 stars 3 forks source link
instructor json json-schema llm local-llm ollama prompting pydantic validation

readme

ollama-instructor

ollama-instructor is a lightweight Python library that provides a convenient wrapper around the Client of the renowned Ollama repository, extending it with validation features for obtaining valid JSON responses from a Large Language Model (LLM). Utilizing Pydantic, ollama-instructor allows users to specify models for JSON schemas and data validation, ensuring that responses from LLMs adhere to the defined schema.

Downloads

Note 1: This library has a native support for the Ollamas Python client. If you want to have more flexibility with other providers like Groq, OpenAI, Perplexity and more, have a look into the great library of instrutor of Jason Lui.

Note 2: This library depends on having Ollama installed and running. For more information, please refer to the official website of Ollama.

Documentation and guides

Examples

Blog

Features

ollama-instructor can help you to get structured and reliable JSON from local LLMs like:

ollama-instructor can be your starting point to build agents by your self. Have full control over agent flows without relying on complex agent framework.

Concept

Concept.png

Find more here: The concept of ollama-instructor

Quick guide

Installation

To install ollama-instructor, run the following command in your terminal:

pip install ollama-instructor

Quick Start

Here are quick examples to get you started with ollama-instructor:

chat completion:

from ollama_instructor.ollama_instructor_client import OllamaInstructorClient
from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

client = OllamaInstructorClient(...)
response = client.chat_completion(
    model='phi3',
    pydantic_model=Person,
    messages=[
        {
            'role': 'user',
            'content': 'Jason is 30 years old.'
        }
    ]
)

print(response['message']['content'])

Output:

{"name": "Jason", "age": 30}

asynchronous chat completion:

from pydantic import BaseModel, ConfigDict
from enum import Enum
from typing import List
import rich
import asyncio

from ollama_instructor.ollama_instructor_client import OllamaInstructorAsyncClient

class Gender(Enum):
    MALE = 'male'
    FEMALE = 'female'

class Person(BaseModel):
    '''
    This model defines a person.
    '''
    name: str
    age: int
    gender: Gender
    friends: List[str] = []

    model_config = ConfigDict(
        extra='forbid'
    )

async def main():
    client = OllamaInstructorAsyncClient(...)
    await client.async_init()  # Important: must call this before using the client

    response = await client.chat_completion(
        model='phi3:instruct',
        pydantic_model=Person,
        messages=[
            {
                'role': 'user',
                'content': 'Jason is 25 years old. Jason loves to play soccer with his friends Nick and Gabriel. His favorite food is pizza.'
            }
        ],
    )
    rich.print(response['message']['content'])

if __name__ == "__main__":
    asyncio.run(main())

chat completion with streaming: (!) Currently broken due to dependency issues with new version of ollama (!)

from ollama_instructor.ollama_instructor_client import OllamaInstructorClient
from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

client = OllamaInstructorClient(...)
response = client.chat_completion_with_stream(
    model='phi3',
    pydantic_model=Person,
    messages=[
        {
            'role': 'user',
            'content': 'Jason is 30 years old.'
        }
    ]
)

for chunk in response:
    print(chunk['message']['content'])

OllamaInstructorClient and OllamaInstructorAsyncClient

The classes OllamaInstructorClient and OllamaInstructorAsyncClient are the main class of the ollama-instructor library. They are the wrapper around the Ollama (async) client and contain the following arguments:

Note: Until versions (v0.4.2) I was working with icecream for debugging. I switched to the logging module.

chat_completion & chat_completion_with_stream

The chat_completion and chat_completion_with_stream methods are the main methods of the library. They are used to generate text completions from a given prompt.

ollama-instructor uses chat_completion and chat_completion_with_stream to expand the chat method of Ollama. For all available arguments of chat see the Ollama documentation.

The following arguments are added to the chat method within chat_completion and chat_completion_with_stream:

Documentation and examples

License

ollama-instructor is released under the MIT License. See the LICENSE file for more details.

Support and Community

If you need help or want to discuss ollama-instructor, feel free to open an issue, a discussion on GitHub or just drop me an email (lennartpollvogt@protonmail.com). I always welcome new ideas of use cases for LLMs and vision models, and would love to cover them in the examples folder. Feel free to discuss them with me via email, issue or discussion section of this repository. 😊