Zero, One and Few Shot Prompting

[realskyrin]

Few-show 可以简单的理解为在发送 prompt 时提前做一个 context mock，通过少量引导可以显著提升 AI 回复质量的技巧。

例如这个查单词小工具

#!/usr/bin/env python3
# This is an English word search assistant.

import json
import requests
import sys
import os

OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY')
OPENAI_BASE_URL = os.environ.get('OPENAI_BASE_URL')

def query(message):
    message = json.dumps(message)

    if len(sys.argv) > 1:
        prompt = "你是一个英语单词查询助手，每当用户发送一个英语单词给你，你都要以固定格式响应用户，" \
                 "如果用户发给你的不是一个单词，回复 'invalid token'"

        response_few_shot_text = "run [/rʌn/]" \
                                 "\n\nn. 奔跑;竞赛;连续的演出\nHe went for a run after work. (他下班后去跑步了)" \
                                 "\n\nv. 奔跑;运行\nI like to run in the park every morning. (我喜欢每天早上在公园里跑步)" \
                                 "\n\nadj. 连续的;流畅的\nThis printer is really fast and runs smoothly. (这台打印机速度非常快，而且运行流畅)"
    else:
        print("This is an English word search assistant")
        print("Usage: qr word")
        return

    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {OPENAI_API_KEY}"
    }

    data = {
        "model": "gpt-3.5-turbo",
        "temperature": 0,
        "top_p": 1,
        "frequency_penalty": 1,
        "presence_penalty": 1,
        "stream": False,
        "messages": [
            {"role": "system", "content": prompt},
            {"role": "user", "content": "run"},
            {"role": "assistant", "content": response_few_shot_text},
            {"role": "user", "content": message}
        ]
    }

    response = requests.post(OPENAI_BASE_URL, headers=headers, data=json.dumps(data))

    answer = json.loads(response.content)["choices"][0]["message"]["content"].strip()
    print("\033[32m---🤖---------\033[0m")
    print(answer)
    print("\033[32m--------------\033[0m")

if __name__ == "__main__":
    query(" ".join(sys.argv[1:]))

我在 request message 中 mock 了 system prompt、user 发送的第一个 prompt 以及 assistant 的一个 response

"messages": [
    {"role": "system", "content": prompt},
    {"role": "user", "content": "run"},
    {"role": "assistant", "content": response_few_shot_text},
    {"role": "user", "content": message}
]

那么我之后再发送任何单词，它都会很规范的返回结果 (返回单词的所有词性并造句)，因为 context 里已经 mock 了它返回的第一个结果，它知道接下来该怎么回复，而无需向它解释过多的东西

如果直接给它发消息作为 prompt 也就是直接一条 user prompt，效果就会差很多，有时候每个词性给出多条例句，或者就只给一个词性的例句

这是 openai-cookbook 中关于 Few shot 的介绍：https://github.com/openai/openai-cookbook/blob/main/techniques_to_improve_reliability.md#few-shot-examples

除此之外，Few-shot 也有助于 AI 进行链式思维以提升结果的准确率，可以应用到很多场景。

我的想法是，可以给每个可以给每个 chat 单独设置 system、user、assistant 这些预设，简单一点的做法是给一个输入框，可以填入整个 request message 部分的 json :

"messages": [
    {"role": "system", "content": sys_prompt},
    {"role": "user", "content": "user_prompt"},
    {"role": "assistant", "content": response_few_shot_text},
    {"role": "user", "content": user_message}
]

请求的时候直接带上这个预设的 message。而且每次 request 仅带这个 message，不需要传入其它上下文，这样一来这个 chat 就变成一个相当稳定且节省 Token 的小工具了。

总之，这个项目非常很赞，只可惜自己不会前端所以以上想法没法直接提 PR，今天现在开始学前端 😂

支持一波

[realskyrin]

畅想，最后可以积木式调用，形成无限扩展。还可以形成指定领域的package，供别人使用。面向GPT编程， 哈哈哈。如果大家觉得这个想法不错，可以一起搞搞事情。

你这个思路很好，事实上很多基于 GPT 做的复杂应用也是这个路子（巨硬家的各种 Copilot），把需求进行无限拆解，拆到能被 GPT 稳定完成的一个个 function，因为 API 是可以并发去 call 的。所以在这些 function call 完之后把结果再组装回来，就能搞出很惊艳的效果。OpenAI 的这条 completion API 真的是有无限可能，值得好好学习。

如果 function 有很多个，并且足够复杂的话，用户能使用的上下文最大长度就会相应的变少。我比较好奇的是怎么尽可能减少这种 function 的 token 占用量？

可能将 Mask（沿用 Yidadaa 对角色提示的定义） 设计成单一职责可以一定程度解决这个问题。但如果把这个问题延伸到 ChatGPT 的 plugin 系统，一个插件中触发某个 api 的 description（描述该 api 何时触发，传入什么，返回什么） 最长为 200 个 token。如果现在有 10 个 plugin，并且每个 plugin 拥有 3 个 api。 那么此时 plugin 的 token 最坏情况下已经达到了 6000， 已经超过 gpt3.5 的限制了。

OpenAI 官方支持了 Function calling, 稳定性终于得到解决，这是一个比 Plugins 更有价值的 Feature https://openai.com/blog/function-calling-and-other-api-updates

@xiaodi007 @ChoKhoOu @Yidadaa

[Yidadaa]

插件功能将于 2023/07 开始开发，这将是本项目的 3.0 版本的重要更新功能。

[realskyrin]

插件功能将于 2023/07 开始开发，这将是本项目的 3.0 版本的重要更新功能。

泰酷辣！！！

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

The plug-in function will be developed in 2023/07, which will be an important update function of the 3.0 version of this project.

Thai cool! ! !

[StainerWei]

您好，我尝试了自己create mask，效果不错。但是我发现一个问题。比如我在笔记本电脑上创建了mask，但是在手机端访问的时候就没有这个mask。或者我的朋友访问的时候，也没有mask。我想知道如何让这个create 的mask可以供所有人使用呢？希望能获得您的帮助，感谢🙏

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Hello, I tried to create mask myself and it worked fine. But I found a problem. For example, I created a mask on my laptop, but I don’t have this mask when I access it on my mobile phone. Or when my friend visited, there was no mask. I want to know how to make this created mask available to everyone? Hope to get your help, thank you 🙏

[daiaji]

Prompt 列表是刚加的功能，后期会允许用户编辑单个对话的预设 prompt，和你说的功能应该是一致的，可以耐心等待。

实际上，我们应该允许编辑所有对话，包括system、user和assistant的内容。在编辑后，可以清除历史摘要并重新发送。

我认为assistant的回答有待改进，希望她能改善用词风格，并纠正一些不恰当或因记忆错误而导致的用词问题。

此外，我自己的回复也可能存在错误，希望能进行修正。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

The Prompt list is a newly added function. Later, users will be allowed to edit the preset prompts of a single conversation. It should be consistent with the function you mentioned, so you can wait patiently.

In fact, we should allow editing of all dialogs, including the contents of system, user, and assistant. After editing, the historical digest can be cleared and resent.

I think the assistant's answer needs to be improved, and I hope she can improve the wording style and correct some words that are inappropriate or caused by memory errors.

Also, there may be errors in my own reply, which I hope will be corrected.

[ifp1090]

请问新的插件可否支持：

1. GPT的文字语音转换，比如用ChatGPT当作英语教练来通过语音练习对话？
2. GPTshare功能可否支持某个页面自定义对话的分享？传统shareGPT的页面分享是整页分享出去的，但分享者也许只需要分享一部分的对话？ 期待答复，谢谢！

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Can the new plugin support:

1. GPT text-to-speech conversion, such as using ChatGPT as an English coach to practice conversations through voice?
2. Can the GPTshare function support the sharing of a custom conversation on a certain page? The page sharing of traditional shareGPT is to share the whole page, but maybe the sharer only needs to share a part of the conversation? Looking forward to reply, thanks!

[ifp1090]

我的意思是，类似「Voice Control for ChatGPT」浏览器插件的文本转语音功能，而不是GPT自己的翻译。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I mean, something like Voice Control for ChatGPT browser plugin's text-to-speech feature, not GPT's own translation.

[2-3-5-7]

您好，我尝试了自己create mask，效果不错。但是我发现一个问题。比如我在笔记本电脑上创建了mask，但是在手机端访问的时候就没有这个mask。或者我的朋友访问的时候，也没有mask。我想知道如何让这个create 的mask可以供所有人使用呢？希望能获得您的帮助，感谢🙏

所有数据都保存在浏览器本地 local storage，目前你想要分享给别人用，可以用 storage explorer 浏览器插件导出成 json 文件，然后在另一个浏览器导入。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Hello, I tried to create mask by myself, and it works well. But I found a problem. For example, I created a mask on my laptop, but I don't have this mask when I access it on my mobile phone. Or when my friend visited, there was no mask. I want to know how to make this created mask available to everyone? Hope to get your help, thank you 🙏

All data is stored in the local storage of the browser. If you want to share it with others, you can use the storage explorer browser plug-in to export it as a json file, and then import it in another browser.

[toyo2333]

我按你代码中的例子，在新版本的功能里添加了 system promp和那个run的问题以及回答例子

但是后续问其他问题，还是会出现，多条例句的情况。

这是什么原因呢？

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

According to the example in your code, I added the system prompt and the run question and answer example in the new version of the function

However, when other questions are asked later, there will still be cases of multiple clauses.

What is the reason?

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Dude, your api is out of money

[ghost]

@yanyongqi911 哥们你API 没钱了啊

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

@yanyongqi911 buddy, your API is out of money

[Yidadaa]

[Tips] 如何使用用户输入模板 How to use Input Template https://github.com/Yidadaa/ChatGPT-Next-Web/issues/2144

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

[Tips] How to use User Input Template How to use Input Template https://github.com/Yidadaa/ChatGPT-Next-Web/issues/2144

[lyh0825]

另外多普及一点，in-context 中的 user、assistant prompt 和 system prompt 一样重要，要不然它们也没必要出现在 API 的示例中。OpenAI 的论文里有很多试验结果已经表明了 Few-shot 可以提升模型解决针对性问题的能力，一个 shot 其实就是 in-context 中的一个 user + assistant 的一问一答，给的例子越多它越准。

比如我上面给出的英语查单词小程序就是 1 shot，针对更复杂的应用场景你可以做 8 shot、25 shot。比较遗憾的是，目前我看到的所有套壳，都没有自定义 Few-shot 的功能，都是 0 shot，可能是觉得这样更方便使用？或者是大家不了解 Few-shot 这个概念，这无疑是把这条强大 API 的能力给绑起来了。我认为一个好的 AI 工具是给 AI 松绑而不是相反。

麻烦可以给出相关的论文link吗

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

In addition, to be more popular, the user, assistant prompt and system prompt in in-context are equally important, otherwise they would not need to appear in the API examples. There are many test results in OpenAI papers that have shown that few-shot can improve the model's ability to solve targeted problems. A shot is actually a question and answer from a user + assistant in-context. The more examples given, the better it will be. allow.

For example, the English word search applet I gave above is 1 shot. For more complex application scenarios, you can do 8 shots or 25 shots. Unfortunately, all the cases I have seen so far do not have the function of customizing Few-shot, and they all have 0 shot. Maybe you think this is more convenient to use? Or maybe everyone doesn’t understand the concept of Few-shot, which undoubtedly limits the capabilities of this powerful API. I think a good AI tool is to loosen the constraints of AI rather than the other way around.

Could you please provide a link to the relevant paper?

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Placement needs English translation Lataie Collins McCall Collins SUPREME Court chambers Laura M Jordan