search

All-Hands-AI / OpenHands

🙌 OpenHands: Code Less, Make More
https://all-hands.dev
MIT License
34.21k stars 3.89k forks source link

Allowing user to specialize a general agent by using prompts from community contributed prompt library #1529

Open xingyaoww opened 6 months ago

xingyaoww commented 6 months ago

What problem or use case are you trying to solve?

Goal: The user can start with a strong and capable general agent from the agenthub but can provide additional information via prompts to "specialize" such agent by giving it domain-specific information. These additional prompts can also be saved to some sort of library and can contribute back to OpenDevin's library (similar to OpenAI's GPTs).

Example: A user wants GPT-4 to perform click house tasks, and GPT-4 is not quite familiar with that. The idea is that, in this case, the user can create a "specialized prompt" containing important snippets of clickhouse documentation with URL links to more of them. The user can then save the prompt, which can just be plain text like CodeAct's example prompt, and submit PR to contribute the prompt into the library.

The next time, when user has clickhouse related request, they can just select "clickhouse-expert" prompt on top of an agent, and the agent will have a better idea of what it is dealing with.

Describe the UX of the solution you'd like

Users can write markdown templates/prompts that easily specialize general agents toward specific tasks (e.g., SWE, Web browsing) -- ideally can do so on the front-end. This is somewhat similar to the current MicroAgent implementation (https://github.com/OpenDevin/OpenDevin/tree/main/agenthub/micro), but not entirely so.

The front-end interface can look something like ChatGPT-Next-Web.

image

Do you have thoughts on the technical implementation?

Describe alternatives you've considered

Additional context

Each "prompt" can be:

  1. A high-level natural language description about what an agent should perform (e.g., you should look through all directories & XXX), and the agent can translate this into what it has in its own action space
  2. AND/OR A more low-level demonstrations (e.g., demonstrating concrete tools to use, like CodeAct's example prompt

If we include (2), we might need an agent-specific implementation for each "prompt" since different agents may handle action spaces differently.

github-actions[bot] commented 5 months ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

neubig commented 2 months ago

@xingyaoww this is maybe mostly fixed by this? https://github.com/All-Hands-AI/OpenHands/pull/3360

xingyaoww commented 2 months ago

I think it will be closed by https://github.com/All-Hands-AI/OpenHands/pull/3511 instead - but we still need FE interface for that too

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] commented 1 week ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.