Provide a Python implementation of TypeChat

[DanielRosenwasser]

This PR brings the basis of a Python implementation into the main TypeChat repository.

For some background, TypeChat has been iterating recently on supporting two other languages/platforms: Python and .NET (chiefly C#). We've been trying to understand what works best as a spec language for LLMs while also providing the most ideal authoring experience.

Usage

At a high level, the APIs are currently very similar to those of our TypeScript implementation, except that our APIs directly take a Python class.

import schema as coffeeshop # schema file

from typechat import TypeChatValidator, TypeChatTranslator

model = ...
validator = TypeChatValidator(coffeeshop.Cart)
translator = TypeChatTranslator(model, validator, coffeeshop.Cart)
print("☕> ", end="", flush=True)
    for line in sys.stdin:
        result = translator.translate(line)
        ...
        print("☕> ", end="", flush=True)

As a reminder: in the above example, we have a validator that is able to ensure any input data conforms to an expected shape. We also have a translator that expresses the expected shape to a given language model.

The biggest difference in usage between TypeScript and Python is that here, coffeeshop.Cart is an actual value being passed in. It's both a runtime value and a declared type - more specifically, a TypedDict. The way it's declared looks like the following:

# schema.py

class UnknownText(TypedDict):
    """
    Represents any text that could not be understood.
    """

    type: Literal["UnknownText"]
    text: str

# ...

class LineItem(TypedDict):
    type: Literal["LineItem"]
    product: Product
    quantity: int

class Cart(TypedDict):
    type: Literal["Cart"]
    items: list[LineItem | UnknownText]

This is in contrast to the original TypeChat validator and translator which acted on TypeScript source. See #147 for an alternative translator that uses Zod.

At the moment, the Python implementation of TypeChat only understands TypedDicts, but I believe we can quickly add support for dataclasses as well. Part of this limitation is simply that the provided validator depends on Pydantic's TypeAdapter to support Python types.

Internals

Background

For some background, something that @umeshma did with C#, and I did with Python in a prototype called Pypechat was to send C# and Python code as spec languages for language models. The concept was to stay as close to the source language as possible throughout. While this generally worked, we found that TypeScript often tends to do bit better with untuned models that are widely-available like GPT-3.5 Turbo. Our experiments have been informal, but we did run through batches of inputs between TypeScript and Python to evaluate success rates and differences. Generally speaking, when given other languages (including C#, Python, and JSON schema), the models we tested on tended to provide more incorrect answers or fill in optional properties with extra information.

We understand that results may differ across models and in the presence of fine-tuning - but in the interest of building something for people that is broadly useful and accessible, Umesh and I have been experimenting versions of TypeChat that use TypeScript as the specification language.

Translating to TypeScript

In this implementation I've built here of TypeChat for Python, our translators perform a conversion of Python type objects into TypeScript declarations.

So if you as a developer define a Person type in Python like the following,

from typing import TypedDict, NotRequired

class Person(TypedDict):
    "This type represents a person."
    name: str
    age: NotRequired[int]

our translator objects will rewrite Person into the following TypeScript declaration before crafting a prompt to a language model:

// This type represents a person.
interface Person {
    name: string;
    age?: number;
}

Under the hood, the translation constructs a TypeScript syntax tree, and then prints that representation out. The machinery to perform this conversion makes some attempt to preserve the way in which Python types were declared, including member ordering, union ordering, and elision of inherited members; however, no attempt is made to preserve the relative ordering of type declarations themselves.

There are some limitations that we may just keep in place - for example, we don't handle name conflicts yet. We may opt to just avoid supporting that, or uniquifying names naively.

There are also some places where I've performed an optimistic translation instead of erroring - but perhaps we should.

# No error on repetition of 'T'.
class Thing[T, T](TypedDict):
    # No error on conflicting annotations.
    attribute: NotRequired[Required[NotRequired[int]]]

But there are other limitations that I would like to see addressed. For example, support for @dataclasses, type parameters gained from Generics, etc. Additionally, we can't take an arbitrary type construct as an input - we only support type aliases and concrete (non-generic) type objects, but that should be easy to implement.

Fixes #38.

[ahejlsberg]

I think we should align the structure of this more closely with the TypeScript version, i.e. have model.py, validator.py, and typechat.py modules with similar structure and functionality. It would help build common understanding and make future maintenance and evolution easier.

[DanielRosenwasser]

I've sent a commit that breaks things into a structure similar to what we have in TypeChat, but in an _internal folder. Everything is re-exported at the top level __init__.py.

There is still no "infer the OpenAI/Azure OpenAI model from settings function", but would rather not block on that.

Let me know if there's anything you'd all rather get in before merging.

[DanielRosenwasser]

In the latest changes I've

• bumped our Pyright dependency
• provided a DefaultOpenAIModel within TypeChat directly
• moved all the Python-to-TS machinery into the _internal/ts_conversion folder
• exposed only a python_type_to_typescript_schema function

[DanielRosenwasser]

The latest change swaps translators and models to be async only. That means there will be a bit of pain out of the box for anyone who built something that is completely synchronous, but otherwise we will have to provide both.

[DanielRosenwasser]

Merging so we can iterate on the main branch, and so people can more-easily clone and try it out. Happy New Year!