Allow one to share system prompts (assistants)

tarasglek commented 7 months ago

We need a better way to share prompts.

I need to share my prompts among my devices
I need to share prompts with other chatcraft users.

One lazy way to do this is to implement it similar to how we did function calling by storing them in github gists, dropping a link to gist into system prompt and replacing contents of system prompt with contents of link dynamically.
Deepset does it like this, it's kinda lame https://github.com/deepset-ai/prompthub/

tarasglek commented 7 months ago

This might be a more important feature than sharing convesations

humphd commented 7 months ago

One challenge of doing this well is making it easy to quickly understand what the system prompt is doing, without having to read and understand the entire thing.

I notice other tools using an avatar and anthropomorphizing the system prompt:

In class the other day, I showed the default ChatCraft system prompt, then made another that was a French speaking Basketball coach. It would be neat if you could somehow package this personality and set of skills into an easily discoverable format that you can pick from with the UI.

humphd commented 7 months ago

I think a cool way to do this would be to support JSON metadata that describes an assistant, which can be stored in the db, stored in R2, posted on the web (e.g., gist) at some URL, etc.

Consider:

{
  "id": "some-unique-id",
  "name": "Senior Software Developer",
  "description": "A senior software developer who specializes in ...",
  "avatar": "https://image.com/path/to/image/to/use/for/assistant.jpg",
  "preferred_model": "openai/gpt-4-0613",
  "system_prompt": "I am a senior software developer, I follow these rules\n\n..."
  "functions": [
    "https://gist.github.com/humphd/74cf88283239c62f53caff5cddf4cfe5",
    "/**\n\nTakes a string and turns it into xyz...\n"
  ]
}

In the above, we define a custom assistant. It includes:

id: a globally unique id, allowing for people to use the same name and not conflict
name: a human-readable name, which we would show in the UI. It might be a title like "Senior Software Developer" or a name like "Carlos" or "DevBot" etc.
description: a human-readable summary of the abilities of the assistant, suitable for display in a "sharing" UI.
avatar: an image to use for the assistant's avatar in the UI
preferred_model: if possible, the model to use. Some users won't have access to a given model, so this is merely a suggestion
system_prompt: the custom system prompt to use
functions: an optional list of functions that the model can choose to call. These can be URLs or TypeScript/JavaScript strings. NOTE: not all models can call functions

We could start by adding a class to support parsing these into something we can use, then add the ability to export an assistant from the current chat and get JSON, then allow sharing it, etc.

humphd commented 7 months ago

In #332, I'm proposing that we support ?assistant=<url>, which could point at the metadata for a custom JSON assistant. We should make that work for the normal app as well as the embed case.

humphd commented 7 months ago

The new @mention thing that ChatGPT just shipped is interesting:

Imagine if you could install the following assistant and give it nickname like @superdev:

{
  "id": "some-unique-id",
  "name": "Senior Software Developer",
  "description": "A senior software developer who specializes in ...",
  "avatar": "https://image.com/path/to/image/to/use/for/assistant.jpg",
  "preferred_model": "openai/gpt-4-0613",
  "system_prompt": "I am a senior software developer, I follow these rules\n\n..."
  "functions": [
    "https://gist.github.com/humphd/74cf88283239c62f53caff5cddf4cfe5",
    "/**\n\nTakes a string and turns it into xyz...\n"
  ]
}

In the db we associate @superdev with this record, and when you include it at the start of your chat, we override the system prompt we send with the chat:

@superdev can you help me with this code ...

Later in the same chat, you ask @mathwizard to help you:

@mathwizard, what's the answer to this problem? ...

Maybe we could also allow these names to be globally available via a URL, so we look up to see if there is one of them in R2. The only problem with that is naming collision among users. But maybe we could do this for ChatCraft owned ones?

tarasglek commented 7 months ago

ooh. this would help with multimodel usecases too. we could ask different models to chime in

tarasglek commented 7 months ago

tried prototyping this flow

I just did a summarization of youtube into json using a special prompt.
Realized that I now wanna use chatcraft coding system prompt and chatgpt3.5 to quickly and cheaply convert it to yaml

so i 1 switched the system prompt and 2 deleted message with youtube import.

this is really powerful. I think when we use @mention we should switch the rest of the convo to a custom model and prompt and provider represented by that model...so all messages after that go to that model.

where it gets interesting is that we then can define custom /export commands which are served by traditional rest apis... publish to blog or run command on linux type mentions

humphd commented 7 months ago

Right, my thinking is that you start out talking to one "assistant" and 3 messages in you can @mention some other assistant, and it switches the system prompt and model. Switching provider is harder, because not everyone uses other providers. Even requesting a specific model is going to fail in lots of cases if your provider doesn't support it--I think it has to be a hint vs. a requirement.

Do you see other things you think we need to add to this serialized format to support that?

{
  "id": "some-unique-id",
  "name": "Senior Software Developer",
  "description": "A senior software developer who specializes in ...",
  "avatar": "https://image.com/path/to/image/to/use/for/assistant.jpg",
  "preferred_model": "openai/gpt-4-0613",
  "system_prompt": "I am a senior software developer, I follow these rules\n\n..."
  "functions": [
    "https://gist.github.com/humphd/74cf88283239c62f53caff5cddf4cfe5",
    "/**\n\nTakes a string and turns it into xyz...\n"
  ]
}

If we use @ for mentioning an assistant, we might need to change @fn and @fn-url to use some other character.

where it gets interesting is that we then can define custom /export commands which are served by traditional rest apis... publish to blog or run command on linux type mentions

Would we do this at the message level or chat level or both? Also we would need to solve auth for remote APIs.

A fast win for this, which wouldn't require more work, would be publish-to-gist if we solved https://github.com/tarasglek/chatcraft.org/issues/381. You could also do a thing where you create a repo, write a Markdown or HTML and let GitHub Pages serve it to publish.

humphd commented 7 months ago

@taras and I discussed this today, and I wanted to capture some notes:

We'd love to be able to extract an "Assistant" from the current chat. So maybe that means you click "Share" on the System Prompt, or it's via some other UI. But the effect is that the system prompt is extracted and somehow in the UI you can set other things like the name, description, avatar, etc. Taras suggested that the System Prompt Message at the top could be updated to be a more complete UI for doing this (e.g., when you click "Edit" on the System Prompt) we could give a more full featured form).
The thing you share (the JSON) can be namespaced by your user name, similar to how we do it now for chats. So I might share an assistant named supermath which would be humphd/supermath, and someone else could pull it into their current chat by doing @humphd/supermath.
We might want to start with having the @ mentions of assistants only operate on the previous message in the chat. The idea being: you're in a chat already, but you want to ask something of this new assistant, but you'll continue back with your regular chat afterward. This raises the question: you do you start a chat with a give assistant vs. pulling one in?
We also discussed whether we could include some commands you can use to tell it to use more context.

There are lots of ideas in here that need to happen in smaller issues/PRs.

humphd commented 6 months ago

Huggingface is getting in on the action too https://twitter.com/xenovacom/status/1753436385367785604:

tarasglek / chatcraft.org

Allow one to share system prompts (assistants) #318