The content of the prompt generator is incomplete

[GeorgeGalway]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.7~0.7.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When there are too many commands, the content returned by the prompt generator is incomplete. This issue has been tested and reproduced multiple times.

✔️ Expected Behavior

I expect the prompt generator to return complete content.

❌ Actual Behavior

The content returned by the prompt generator is not exactly what I want, especially the second half, where the examples are incomplete. This leads to poor output quality.

[dosubot[bot]]

To fix the issue where the prompt generator returns incomplete content when there are too many commands in Dify version 0.6.7~0.7.3, you should set Stop Sequences. This will inform the LLM to stop generating text at specific points, preventing it from producing incomplete content.

Here’s how you can set Stop Sequences:

1. Identify the specific words, phrases, or characters that should signal the LLM to stop generating text. For example, if you have a prompt like:

   Human1: What color is the sky?
   Assistant1: The sky is blue.
   Human1: What color is fire?
   Assistant1: Fire is red.
   Human1: What color is soil?
   Assistant1: 

2. In the model parameters' Stop Sequences, input Human1:, and press the "Tab" key.

   This way, the LLM will only respond with one sentence:

   Assistant1: Soil is yellow.

   And will not generate additional dialogue (i.e., the LLM will stop generating content before reaching the next "Human1:").

By setting the appropriate stop sequences, you can control the output length and ensure that the content generated is complete and as expected ^[1].

To continue talking to Dosu, mention @dosu.

[Sakura4036]

You can generate longer prompts by setting the env environment variable PROMPT_GENERATION_MAX_TOKENS in .env file, the default value is 512.

[GeorgeGalway]

You can generate longer prompts by setting the env environment variable PROMPT_GENERATION_MAX_TOKENS in .env file, the default value is 512.

However, configuring it in the env didn’t work, including placing it in the api/.env file and docker/.env file.

[Sakura4036]

I set PROMPT_GENERATION_MAX_TOKENS=2048 works for me. Maybe you should check your docker config file.

• PROMPT_GENERATION_MAX_TOKENS=2048
• PROMPT_GENERATION_MAX_TOKENS=256

Although the generated prompts are shorter and incomplete, they are not the maximum length I set, which may be a problem.

[GeorgeGalway]

我设置PROMPT_GENERATION_MAX_TOKENS=2048成功了。也许你应该检查一下你的 docker 配置文件。

• PROMPT_GENERATION_MAX_TOKENS=2048
• PROMPT_GENERATION_MAX_TOKENS=256

虽然生成的提示比较短而且不完整，但是都不是我设定的最大长度，这可能是一个问题。

Is your env file located at /docker/.env or /api/.env?

[Sakura4036]

Self Hosted (Source) depolyment， so i use api/.env file for configuration

[GeorgeGalway]

Self Hosted (Source) depolyment， so i use api/.env file for configuration

Thanks for getting back to me. Self Hosted (Docker)，regardless of whether it's in /api/.env or /docker/.env, the PROMPT_GENERATION_MAX_TOKENS parameter isn't having any effect.

[Sakura4036]

@crazywoola Could you offer some help?

[nadirvishun]

Self Hosted (Source) depolyment， so i use api/.env file for configuration

Thanks for getting back to me. Self Hosted (Docker)，regardless of whether it's in /api/.env or /docker/.env, the PROMPT_GENERATION_MAX_TOKENS parameter isn't having any effect.

It seems that we also need to configure x-shared-env: in docker-compose.yaml to include PROMPT_GENERATION_MAX_TOKENS, because docker/.env is on the host machine and does not affect the docker environment. It can only take effect after being transformed through the environment in docker-compose.yaml. However, both the .env file and docker-compose.yaml in the docker self hosted lack the PROMPT_GENERATION_MAX_TOKENS parameter, which is a bug.