search

transitive-bullshit / OpenOpenAI

Self-hosted version of OpenAI’s new stateful Assistants API
MIT License
509 stars 44 forks source link
assistants gpts openai openai-api self-hosted

readme

OpenOpenAI

Example usage

Build Status MIT License Prettier Code Formatting

Intro

This project is a self-hosted version of OpenAI's new stateful Assistants API. 💪

All API route definitions and types are 100% auto-generated from OpenAI's official OpenAPI spec, so all it takes to switch between the official API and your custom API is changing the baseURL. 🤯

This means that all API parameters, responses, and types are wire-compatible with the official OpenAI API, and the fact that they're auto-generated means that it will be relatively easy to keep them in sync over time.

Here's an example using the official Node.js openai package:

import OpenAI from 'openai'

// The only difference is the `baseURL` pointing to your custom API server 🔥
const openai = new OpenAI({
  baseURL: 'http://localhost:3000'
})

// Since the custom API is spec-compliant with OpenAI, you can use the sdk normally 💯
const assistant = await openai.beta.assistants.create({
  model: 'gpt-4-1106-preview',
  instructions: 'You are a helpful assistant.'
})
Python example Here's the same example using the official Python `openai` package: ```py from openai import OpenAI client = OpenAI( base_url: "http://localhost:3000" ) # Now you can use the sdk normally! # (only file and beta assistant resources are currently supported) # You can even switch back and forth between the official and custom APIs! assistant = client.beta.assistants.create( model="gpt-4-1106-preview", description="You are a helpful assistant." ) ```

Note that this project is not meant to be a full recreation of the entire OpenAI API. Rather, it is focused only on the stateful portions of the new Assistants API. The following resource types are supported:

See the official OpenAI Assistants Guide for more info on how Assistants work.

Why?

Being able to run your own, custom OpenAI Assistants that are 100% compatible with the official OpenAI Assistants unlocks all sorts of useful possibilities:

Most importantly, if the OpenAI "GPT Store" ends up gaining traction with ChatGPT's 100M weekly active users, then the ability to reliably run, debug, and customize OpenAI-compatible Assistants will end up being incredibly important in the future.

I could even imagine a future Assistant store which is fully compatible with OpenAI's GPTs, but instead of relying on OpenAI as the gatekeeper, it could be fully or partially decentralized. 💯

Stack

Development

Prerequisites:

Install deps:

pnpm install

Generate the prisma types locally:

pnpm generate

Environment Variables

cp .env.example .env

Services

The app is composed of two services: a RESTful API server and an async task runner. Both services are stateless and can be scaled horizontally.

There are two ways to run these services locally. The quickest way is via tsx:

# Start the REST API server in one shell
npx tsx src/server

# Start an async task queue runner in another shell
npx tsx src/runner

Alternatively, you can transpile the source TS to JS first, which is preferred for running in production:

pnpm build

# Start the REST API server in one shell
npx tsx dist/server

# Start an async task queue runner in another shell
npx tsx dist/runner

E2E Examples

Custom Function Example

This example contains an end-to-end assistant script which uses a custom get_weather function.

You can run it using the official openai client for Node.js against the default OpenAI API hosted at https://api.openai.com/v1.

npx tsx e2e

To run the same test suite against your local API, you can run:

OPENAI_API_BASE_URL='http://127.0.0.1:3000' npx tsx e2e

It's pretty cool to see both test suites running the exact same Assistants code using the official OpenAI Node.js client – without any noticeable differences between the two versions. Huzzah! 🥳

Retrieval Tool Example

This example contains an end-to-end assistant script which uses the built-in retrieval tool with this readme.md file as an attachment.

You can run it using the official openai client for Node.js against the default OpenAI API hosted at https://api.openai.com/v1.

npx tsx e2e/retrieval.ts

To run the same test suite against your local API, you can run:

OPENAI_API_BASE_URL='http://127.0.0.1:3000' npx tsx e2e/retrieval.ts

The output will likely differ slightly due to differences in OpenAI's built-in retrieval implementation and our default, naive retrieval implementation.

Note that the current retrieval implementation only support text files like text/plain and markdown, as no preprocessing or conversions are done at the moment. We also use a very naive retrieval method at the moment which always returns the full file contents as opposed to pre-processing them and only returning the most semantically relevant chunks. See this issue for more info.

Server routes

GET       /files
POST      /files
DELETE    /files/:file_id
GET       /files/:file_id
GET       /files/:file_id/content
GET       /assistants
POST      /assistants
GET       /assistants/:assistant_id
POST      /assistants/:assistant_id
DELETE    /assistants/:assistant_id
GET       /assistants/:assistant_id/files
GET       /assistants/:assistant_id/files
POST      /assistants/:assistant_id/files
DELETE    /assistants/:assistant_id/files/:file_id
GET       /assistants/:assistant_id/files/:file_id
POST      /threads
GET       /threads/:thread_id
POST      /threads/:thread_id
DELETE    /threads/:thread_id
GET       /threads/:thread_id/messages
POST      /threads/:thread_id/messages
GET       /threads/:thread_id/messages/:message_id
POST      /threads/:thread_id/messages/:message_id
GET       /threads/:thread_id/messages/:message_id/files
GET       /threads/:thread_id/messages/:message_id/files/:file_id
GET       /threads/:thread_id/runs
POST      /threads/runs
POST      /threads/:thread_id/runs
GET       /threads/:thread_id/runs/:run_id
POST      /threads/:thread_id/runs/:run_id
POST      /threads/:thread_id/runs/:run_id/submit_tool_outputs
POST      /threads/:thread_id/runs/:run_id/cancel
GET       /threads/:thread_id/runs/:run_id/steps
GET       /threads/:thread_id/runs/:run_id/steps/:step_id
GET       /openapi

You can view the server's auto-generated openapi spec by running the server and then visiting http://127.0.0.1:3000/openapi

TODO

Status: All API routes have been tested side-by-side with the official OpenAI API and are working as expected. The only missing features at the moment are support for the built-in code_interpreter tool (issue) and support for non-text files with the built-in retrieval tool (issue). All other functionality should be fully supported and wire-compatible with the official API.

TODO:

License

MIT © Travis Fischer

If you found this project useful, please consider sponsoring me or following me on twitter twitter