coleam00 / bolt.new-any-llm

Prompt, run, edit, and deploy full-stack web applications using any LLM you want!
https://bolt.new
MIT License
3.89k stars 1.59k forks source link

Model forgets instructions: Just resend system prompt #299

Open smarthiesbs opened 6 days ago

smarthiesbs commented 6 days ago

Is your feature request related to a problem? Please describe:

The use of custom models sometimes leads to problems with prompt adherence and context size. The model then forgets how to create files and what the system commands for starting the web server should look like. When the source code is only displayed in the chat window and no longer appears in the code area, you don't have many chances to bring back your model into life.

Describe the solution you'd like:

But in my experience the following solution often works: Write the system prompt ("You are Bolt, an expert of...") again in the chat window and ask the model to follow these instructions. And suddenly the file creation works again. Of course, there is no 100% guarantee, but in my experience this is a good workaround if you are running models that quickly forget their instructions due to small context windows. I could imagine a “Resend System Prompt” button which you could automate this refresh (only the area up to the examples section is actually needed).

Describe alternatives you've considered:

With a shorter system prompt, you could of course delay the problematic moment. This approach should therefore also be pursued.

Additional context:

Before (model only quotes source code to the chat window): image

After resending system prompt (model again uses the artifacts and creates the files in the workbench):

image

image

Warning

If the model becomes forgetful, you should definitely download or sync the files. There is then a great risk of losing all progress. In this case, it may be necessary to reinsert content manually. This is of course unacceptable in a production environment, but as long as ottodev does not provide a backup, snapshot or caching function, there is no other solution.