Zod validation support, switch to Fetch API, and refactoring

[ahejlsberg]

This PR introduces the following changes:

• Support for validation using Zod, a popular TypeScript-first schema validation library.
• Switch to using the JavaScript Fetch API for LLM access and remove dependency on axios package.
• Refactoring of certain functions and module names.

Zod support

The new Zod support is useful in applications that already incorporate JSON validation and/or dynamically change schema based on interaction history, databases, or other information sources. Also, Zod is significantly lighter weight than incorporating the TypeScript compiler for validation purposes.

Even though validation of JSON responses can now be performed by the Zod validator, schemas are still communicated to the LLM as TypeScript type definitions. The new typechat/zod validator automatically converts Zod schema to TypeScript source code suitable for inclusion in LLM prompts (and the conversion is available in a helper function for other purposes). In our experience, using TypeScript to describe schema to LLMs produces better outcomes and consumes fewer tokens than using JSON Schema.

Two new examples, examples/coffeeShop-zod and examples/sentiment-zod, demonstrate the new Zod support. These are Zod equivalents of the corresponding examples that use the TypeScript compiler for validation.

An example of a simple Zod schema for determining the sentiment of a some user input.:

import { createLanguageModel, createJsonTranslator, processRequests } from "typechat";
import { createZodJsonValidator } from "typechat/zod";
import { z } from "zod";

const SentimentResponse = z.object({
    sentiment: z.enum(["negative", "neutral", "positive"]).describe("The sentiment of the text")
});

const SentimentSchema = {
    SentimentResponse
};

const model = createLanguageModel(process.env);
const validator = createZodJsonValidator(SentimentSchema, "SentimentResponse");
const translator = createJsonTranslator(model, validator);

// Process requests interactively or from the input file specified on the command line
processRequests("😀> ", undefined, async (request) => {
    const response = await translator.translate(request);
    if (!response.success) {
        console.log(response.message);
        return;
    }
    console.log(`The sentiment is ${response.data.sentiment}`);
});

In addition to the createZodJsonValidator function, the new typechat/zod module exports a convenient getZodSchemaAsTypeScript function that can be used to convert Zod schemas into TypeScript source code.

Switch to Fetch API

The implementation has switched to using the JavaScript Fetch API instead of depending on the axios package. This helps reduce bundle sizes with no effect on client applications.

Refactoring

The implementation has been refactored as follows:

• Validators are now in the typechat/ts and typechat/zod modules (instead of the main typechat module).
• The createJsonTranslator function now takes a validator parameter instead of actually creating the validator.
• The TypeChatJsonValidator interface has been changed to have methods instead of properties for accessing the schema text and target type name, and the validate method has been changed to take an object instead of a string parameter.

These are breaking changes, but quite easy to adopt.

[steveluc]

👍

[DanielRosenwasser]

If we're good with the current state of things, can you just pull from main, run an npm install, and update the package-lock.json?