search

Significant-Gravitas / AutoGPT-Code-Ability

🖥️ AutoGPT's Coding Ability - empowering everyone to build software using AI
MIT License
109 stars 34 forks source link

Raise Enhancements up to the level of the prompts #226

Closed ntindle closed 4 months ago

ntindle commented 4 months ago

Raise error messages to the prompt

linear[bot] commented 4 months ago

AGPT-695

Swiftyos commented 4 months ago

It would probably be a good idea to start measuring the impact of our changes before merging them in.

I suspect we want to be trying to minimise the length of our prompts as much as possible

Torantulino commented 4 months ago

This should be a great change, but we have to be careful that this doesn't flood the prompt with knock-on error messages and hide the real error which is causing the issue.

How have you found the success rate of this vs the existing error enhancements?

Curious to see some prompts this has constructed.

ntindle commented 4 months ago

/benchmark

Torantulino commented 4 months ago

I suspect we want to be trying to minimise the length of our prompts as much as possible

We certainly want to keep prompts small to avoid the "needle in the haystack" problem, and general degradation of performance that comes with long prompts.

However inserting errors with small documentation snippets like this should in theory be a great use of that token budget in my opinion.