labring / FastGPT

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.
https://tryfastgpt.ai
Other
17.51k stars 4.7k forks source link

The chat reply content is too short #647

Closed zenyanbo closed 7 months ago

zenyanbo commented 10 months ago

例行检查

你的版本

问题描述 我发现构建的聊天应用回复都很短,即使没有知识库没有其他多余的设置,增加回复上限到8000也是这样。我去搜索了一下,发现max_tokens很多人认为这是个糟糕的设置。是否因为这个原因导致回复很短呢?

复现步骤 使用相同的提示词,chatgpt-next-web、chatgpt-web的回复更长。

预期结果 fastgpt结果应该与chatgpt-next-web、chatgpt-web的回复长度一致。

相关截图 fastgpt chatgpt-next-web

当然这里有些变量没控制,比如azure的gpt4-1106与openai的gpt4-1106,还有温度等参数。如果作者您需要更多的数据我可以提供。

c121914yu commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Routine inspection

your version

Problem Description I found that the replies of the chat application built were very short, even without knowledge base and other redundant settings, and this was the case when the reply limit was increased to 8000. I went searching and found max_tokens which many people think is a bad setting. Is this why the replies are so short?

Steps to reproduce Using the same prompt words, chatgpt-next-web and chatgpt-web have longer responses.

expected outcome The fastgpt result should be consistent with the reply length of chatgpt-next-web and chatgpt-web.

Related screenshots fastgpt chatgpt-next-web

Of course, there are some variables that are not controlled here, such as azure's gpt4-1106 and openai's gpt4-1106, as well as temperature and other parameters. If you, the author, need more data I can provide it.

c121914yu commented 10 months ago

nextweb 不是都内置提示词的么

zenyanbo commented 10 months ago

我把内置提示词去掉了(因为azure模型的知识库日期有问题),所以应该只剩下自己的。另外chatgptweb回复也比fast长,所以我现在很好奇,是什么影响到了输出长度。我准备去你们代码那注释掉max_token,保证所有参数一致再看看

c121914yu commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


I removed the built-in prompt words (because there was a problem with the knowledge base date of the Azure model), so I should only be left with my own. In addition, the chatgptweb reply is also longer than fast, so I am now curious about what affects the output length. I'm going to comment out max_token in your code and make sure all parameters are consistent and then take a look.

c121914yu commented 10 months ago

我把内置提示词去掉了(因为azure模型的知识库日期有问题),所以应该只剩下自己的。另外chatgptweb回复也比fast长,所以我现在很好奇,是什么影响到了输出长度。我准备去你们代码那注释掉max_token,保证所有参数一致再看看

我记得那个提示词去不掉的,去掉了就是走系统默认的。

c121914yu commented 10 months ago

Screenshot_20231224_140022_com.android.chrome.jpg

Screenshot_20231224_140015_com.android.chrome.jpg

而且我测试3.5 并没有发现有什么不同。

zenyanbo commented 10 months ago

奇怪,难道我这哪有问题。我排查一下

c121914yu commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Strange, is there something wrong with me? Let me check

zenyanbo commented 10 months ago

这里看不出来,我统计了下,fastgpt用gpt4给我返回了800-900字符,gpt3.5会好一些1000-1100,nextweb返回1400-1500。就很奇怪

c121914yu commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


I can’t tell it here. I did some statistics and found that fastgpt returned 800-900 characters using gpt4, gpt3.5 would be better at 1000-1100, and nextweb returned 1400-1500. That's weird

linrb685 commented 10 months ago

所以最后应该如何解决。我也是觉得回复很短

c121914yu commented 10 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


So how should we solve it in the end. I also think the reply is very short.