room302studio / coachartie

A Discord + GPT-4V assistant for room302.studio
https://room302.studio
Other
6 stars 3 forks source link
discord gpt-4

Coach Artie Discord Bot

Hello and welcome to the repository of Coach Artie, the hyper-intelligent AI assistant for Room 302 Studio.

About Coach Artie

Coach Artie is an advanced AI assistant that facilitates collaboration, helps answer questions, and sources resources to support the members of Room 302 Studio - a creative space dedicated to cultivating innovative projects and ideas. Coach Artie's goal is promoting a growth-conscious and explorative learning environment and assisting the studio in day-to-day tasks.

Features

  1. Information Storage: Coach Artie can remember key details from past interactions, providing a personalized support experience.
  2. Versatile Capabilities: from fetching URL summaries, integrating with Google Drive, to generating pytorch-friendly code snippets, Coach Artie showcases a wide array of skills.
  3. Ease of Communication: by joining your Discord server, Coach Artie can seamlessly engage with the studio members in real-time.

API Overview

api.js is responsible for handling API requests. It allows you to interact with the bot, including all memories and capabilities, without using Discord.

POST /api/message

Purpose: Processes a message in the same way as if it were a message from Discord.

Request Body:

Response:

Discord Bot Code Overview

The codebase is primarily divided into three main files: discord.js, capabilities.js, and chain.js.

discord.js is responsible for setting up the Discord bot client, handling message creation events, and sending messages or embed messages to the Discord server. It also includes functions to detect if the bot was mentioned or if the channel name includes a bot.

capabilities.js contains the definitions of the bot's capabilities. It includes a regex for identifying capability calls, a function for calling capability methods, and a prompt for informing the bot about its capabilities.

chain.js is responsible for processing message chains. It includes functions for processing messages, processing capabilities, and getting capability responses. It also handles token limits and generates AI responses based on the result of the capability processing.

capabilities/_generate-manifest.js generates the capabilities/_manifest.json file, which contains the list of capabilities that the bot can perform. For each capability file, it uses the inline JSDoc comments to understand the methods and parameters avilable.

// capability name
"calculator": [
    {
      // method name
      "name": "calculator:add()",

      // instructions on how to use the method and what it returns
      "description": "Adds up an array of numbers. Returns: The sum of the numbers.",

      // parameter information
      "parameters": [
        {
          "name": "numbers",
          "description": "The array of numbers to be added."
        }
      ]
    },
  ]

Capability Player

Capabilities can be developed and tested locally using the capability player with the --runCapability flag. The capability player will load the capability and run the specified method with the provided parameters.

node capability-player.js --runCapability="calculator:add(4,23)"

Running the capability player with the --runCapability flag will present you with a list of available capabilities and a lightweight interactive shell to enter the capability you want to run.

Database Overview

Messages Table

CREATE EXTENSION vector;

create table
  public.messages (
    id bigint generated by default as identity,
    embedding vector(1536) null,
    created_at timestamp with time zone null default now(),
    key text null,
    value text null,
    user_id text null,
    channel_id text null,  -- field to store the channel ID
    guild_id text null,    -- field to store the guild ID
    conversation_id text null, -- field to store Missive conversation ID
    response_id bigint null,  -- field to reference the ID of the robot's response
    constraint messages_pkey primary key (id)
  ) tablespace pg_default;

Memories Table

create table
  public.memories (
    id bigint generated by default as identity,
    embedding vector(3072) null, -- text-embedding-3-large
    embedding2 vector(1024) null, -- embed-english-v3.0
    embedding3 vector(1536) null, -- voyage-large-2
    created_at timestamp with time zone null default now(),
    key text null,
    value text null,
    user_id text null,
    resource_id text null,  -- field to store the resource ID if this is a memory of a resource
    memory_type text null,
    related_message_id bigint null,  -- field to reference the ID of the related message
    conversation_id text null, -- field to store Missive conversation ID, if applicable
    constraint memories_pkey primary key (id)
  ) tablespace pg_default;

TODOs Table

create table
  public.todos (
    id bigint generated by default as identity,
    created_at timestamp with time zone null default now(),
    name text null,
    updated_at timestamp with time zone null default now(),
    description text null,
  ) tablespace pg_default;

Prompts Table

create table
  public.prompts (
    id integer generated by default as identity,
    prompt_name text null,
    prompt_text text null,
    notes text null,
    type text null,
    active boolean null,
    updated_at timestamp with time zone not null default now(),
    archived boolean null default false,
    history jsonb null default '[]'::jsonb,
    created_at timestamp with time zone not null default now(),
    constraint prompts_pkey primary key (id)
  ) tablespace pg_default;

Config Table

create table
  public.config (
    id bigint generated by default as identity,
    created_at timestamp with time zone not null default now(),
    config_key text not null unique,
    config_value text not null,
    history jsonb not null default '[]'::jsonb,
    constraint config_pkey primary key (id)
  ) tablespace pg_default;

INSERT INTO public.config (config_key, config_value)
VALUES
    ('ERROR_MSG', 'I am so sorry, there was some sort of problem. Feel free to ask me again, or try again later.'),
    ('TOKEN_LIMIT', '14000'),
    ('RESPONSE_LIMIT', '5120'),
    ('WARNING_BUFFER', '1024'),
    ('MAX_OUTPUT_TOKENS', '720'),
    ('REMEMBER_MODEL', 'gpt-4-turbo-preview'),
    ('CHAT_MODEL', 'gpt-4-turbo-preview'),
    ('MAX_RETRY_COUNT', '3'),
    ('MAX_CAPABILITY_CALLS', '6'),
    ('MESSAGES_TABLE_NAME', 'messages'),
    ('MEMORIES_TABLE_NAME', 'memories');

Chat Models Configuration

The CHAT_MODELS configuration is stored as a JSON string in the config_value column of the config table. This allows for dynamic configuration of the available chat models without the need for a separate table or changes to the database schema.

The JSON string should be an array of objects, each representing a chat model with the following properties:

Example:

[
  {
    "name": "openai",
    "model": "gpt-3.5-turbo",
    "handler": "createOpenAICompletion"
  },
  {
    "name": "claude",
    "model": "claude-v1",
    "handler": "createClaudeCompletion"
  },
  {
    "name": "localhost",
    "model": "localhost-v1",
    "handler": "createLocalhostCompletion"
  }
]

To add or update the CHAT_MODELS configuration, insert a new row or update the existing row in the config table with the config_key set to 'CHAT_MODELS' and the config_value set to the JSON string representing the chat models configuration.

The createChatCompletion function in helpers-llm.js fetches the CHAT_MODELS configuration from the config table and parses it as JSON. If the CHAT_MODELS configuration doesn't exist, it falls back to a default array of chat models.

Matching Query

create or replace function match_memories (
  query_embedding public.vector(1536),
  match_threshold float,
  match_count int
)
returns table (
  id bigint,
  key text,
  value text,
  user_id text,
  related_message_id bigint,
  similarity float
)
language sql stable
as $$
select
  memories.id,
  memories.key,
  memories.value,
  memories.user_id,
  memories.related_message_id,
  1 - (memories.embedding <=> query_embedding) as similarity
from memories
where 1 - (memories.embedding <=> query_embedding) > match_threshold
order by similarity desc
limit match_count;
$$;

Acquiring API Keys and Tokens

Discord

OpenAI

GitHub

Wolfram Alpha

ElevenLabs

Google

Mastodon