Add support for options

[simonw]

These are available as an OpenAI schema from Replicate, which I could parse and turn into LLM Options.

[simonw]

A fun twist here is that usually Options are the same for every instance of a model, hence they are defined as a Model.Options inner class - but here they will differ for different Replicate models.

[simonw]

Here's the relevant schema from my fixture: https://github.com/simonw/llm-replicate/blob/35933fe8286f19277b9ac5f7ec820257e8b8e344/tests/replicate-flan-t5-xl.json#L261-L317

[simonw]

It would be really neat if Pydantic could handle converting this into a class for me.

[simonw]

Couldn't find a way to do it with Pydantic, but doing it by hand shouldn't be too hard - it looks like it's mainly things like this:

    "top_p": { 
       "type": "number", 
       "title": "Top P", 
       "default": 1, 
       "maximum": 1, 
       "minimum": 0.01, 
       "x-order": 3, 
       "description": "When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens" 
     }, 

in a dictionary at:

details['latest_version']['openapi_schema']['components']['schemas']['Input']['properties']

[simonw]

ChatGPT: https://chat.openai.com/share/d03dfa43-5042-41d1-9945-679d4a681ac5

from pydantic import create_model, Field, conint, confloat
from typing import Optional

# JSON schema
schema = {
    "type": "number",
    "title": "Top P",
    "default": 1,
    "maximum": 1,
    "minimum": 0.01,
    "x-order": 3,
    "description": "When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens"
}

# Set appropriate Python types based on the schema's type
type_mapping = {
    "number": confloat,
    "integer": conint,
    "string": str,
    "boolean": bool,
    "object": dict,
    "array": list,
}

# Dynamically create the Pydantic model
TopPModel = create_model(
    "TopPModel",
    top_p=(Optional[type_mapping[schema['type']](gt=schema.get('minimum'), lt=schema.get('maximum'))], Field(
        default=schema.get('default'),
        description=schema.get('description'),
    )),
)

# Now we can use the TopPModel
instance = TopPModel(top_p=0.5)
print(instance)