Enhance prompts to ask the LLM to plan things ahead

[tobitege]

I find it helpful to include in my initial prompt to first ask the LLM to make a detailed plan about the task and then follow its steps.

In this context, it'd be to create steps for whatever is set out in an issue's description.

Some rejection/exclusion criteria could minimize the LLM run away with hours of work without an unreachable goal. In case of rejection, the LLM could be allowed to post a GitHub comment for that issue with its reasoning that lead to rejection.

It first needs to e.g. recognize whether the issue is actually about the repo itself and not about e.g. ideas how other software is doing something or to integrate a totally different software that would be in the "huge effort" category (in our understanding).

In addition, have it write the plan to an actual text file (with issue number in it), so it can reference it throughout the process ("Let's deal with step 6..."). In case of failure (or hard restrictions/limits), it could be asked to resume on it later.

TL;DR tell the LLM to not go wild, but make a plan first.

[codegefluester]

Did you try adding a .openhands_instructions file in the root folder of your repository? That file appears to be read and injected into the initial prompt.

[tobitege]

That file is only used by the openhands resolver package and was only recently added (2 weeks ago). It is not being read by OH or its agents, last I checked

[codegefluester]

That file is only used by the openhands resolver package and was only recently added (2 weeks ago). It is not being read by OH or its agents, last I checked

I ran a resolver GitHub action just today and it did read the file + injected it into the task description for the agent, verified by looking at the logs.

[tobitege]

Yes, that's what I wrote: only the resolver uses it, not if you use OH itself.

[codegefluester]

Yes, that's what I wrote: only the resolver uses it, not if you use OH itself.

Ah sorry, didn't parse that correctly