search

Significant-Gravitas / AutoGPT

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
https://agpt.co
Other
167.73k stars 44.28k forks source link

Procedural Memory / Tool building #4214

Closed Boostrix closed 7 months ago

Boostrix commented 1 year ago

Duplicates

Summary 💡

Mentioned on Discord: https://discord.com/channels/1092243196446249134/1100458518693281873/1100472662452600972

Obviously, AutoGPT can already build custom tools (scripts) - however, it might make sense to formalize this:

The idea discussed on discord was to allow it to grow a library of tools - so that it can extend itself over time and reuse its own tools for different projects and purposes (which would imply that tools would not be workspace specific).

That sounds actually like a good idea to me - and it would help with the issue of workspaces lacking persistence and of agents not being able to suspend/resume currently.

This way, at least, created tools could be reused.

A straightforward step would be to add a new command "create_tool" that then takes some specs for the tool to build, and which will add the tool building process to a JSON file that lives outside the workspace. That way, the agent could reuse these tools for future work.

We could add a corresponding list_tools command that would then traverse the json file/tool directory to list all tools and treat them analogous to "commands", using the I/O specs previously defined.

So, basically the agent would get new commands to:

This would not be doing much more than use create_script / execute_python respectively - except that it could go through a handful of steps that involve gathering data/specs and serializing those to some file on disk (think JSON).

Once a tool is finished, it would show up and could be used by agents for future tasks

Over time, this could be adapted to also take a "type" parameter, where type could be:

This would cover most use-cases we've previously seen in #15, and it would be a first step towards self-improvement.

Examples 🌈

An agent needing to scrape a website as part of its tasks should create a generic and reusable tool as part of a "tools workspace" where it may store stuff that could come in handy later on. The main objective here being to save time/resources, and update a list of built-in tools that were previously created/tested and that are functional.

Future instances having to use similar tools could either use those, or -by looking at the code inside the "tools workspace"- come up with custom/adapted versions of the tool in question. The idea would be to do recursive test-driven-development to adapt existing tools as needed. This may involve supporting versioning, so that new/adapted tools use a different version number.

Tools could by convention provide a query parameter to learn more about their capabilities - so that an ai agent could run a tool to learn more about it (think --help --verbose but for BIFs)

The tool builder would be provided with a template for building new tools, so that a certain struture is used by all tools. Tools would contain sufficient comments/docstrings to provide context for self-modification. And each functional part of a tool would be required to come with tests to ensure that a tool is functioning, but also to identify areas that are not yet functional - which could then be used by an agent to narrow down/focus itself to "finish" a tool.

Motivation 🔦

autogpt tends to create a ton of scripts to accomplish its tasks - often, these are only useful to the very instance that originally created the script - other instances will usually not know how to use such tools, even if they encounter the corresponding scripts.

It would make sense for autogpt to specifically create tools with a focus on being able to reuse those, this entails encoding input/output requirements (parameters / return values) and the current state of a tool's development. Once a tool is "finished" it would show up in the library of tools.

It would be a first step towards autogpt being able to extend itself, i.e. to self-improve in a reusable fashion. Without that being specific to a single agent/project [session].

We could prime the "tool builder" to focus on coming up with generic tools that may be of use in the future, and that may also be of use to others.

It is likely that this way ANY autogpt instance will at least partially work toward a common goal, so that future instances have less work to do, which will save time/resources (API tokens)

The development process could go through a handful of stages or levels:

For starters, the tool builder could get restriction for each associated file to be restricted to a size that fits the context window, e.g. to split up each "tool" into N python files that don't need more than ~3000 tokens. N could be made configurable. Which should improve the odds of the tool builder succeeding at its work.

ether8unny commented 1 year ago

https://twitter.com/tianle_cai/status/1662988435114852352?s=46&t=b7QZe5_0_rjTrflh249NyQ an interesting method for building tools

Boostrix commented 1 year ago

We've also been talking about the notion of repeatable workflows: https://discord.com/channels/1092243196446249134/1097204317125091328/1114495196508463139

Here, a workflow would be a prerecorded/precreated sequence (list) of steps to accomplish a goal (objective) by allowing these to be parameterized when invoked and handling most work agent-side with little/zero LLM involvement

bfalans commented 1 year ago

@Boostrix we should definitely have a chat. Am home in about 2 hours

carlosbaraza commented 1 year ago

This would be interesting. Many papers talking about how tools can extend the capabilities of agents. I was thinking on building a package manager for tools, a repository an agent could search to find relevant tools, so the agent doesn't need to build a tool twice

Boostrix commented 1 year ago

that sounds like an awesome idea, but I believe it would be a little premature at this point, no ? my thinking was, that we need to expose/generalize the command manager interface so that new commands can be dynamically registered by mapping those to standalone python scripts.

And like I said previously, I was thinking that these scripts would live outside the workspace, e.g. some sort of "super workspace" that is to be shared by different agents/instances.

Tools would then by convention get a ton of tests to ensure that they can be incrementally tested/developed and updated as needed. Also, by convention, each tool could be inspected so that its source code can be used as a template for other tools.

Also, the original idea was brought up on Discord, so we may want to read up on the whole idea there. A package manager would definitely be interesting, especially given the size/acitvity of this community, but in my opinion a little premature given the current state of affairs ?

On the other hand, keep the ideas coming and we can take it from there - I am sure there is more than just one Python module/framework to implemented a decentralized package manager. Given the size of the community, this could even work in a p2p fashion

rihp commented 1 year ago

We want to develop a prototype plugin for AutoGPT to extend its functionality, and wanted to begin with a simple survey before we dive in developing.

We would like to get maybe 25 responses, and we would definitely share back the findings with the AutoGPT community, while using the results to better shape our contribution

Here is the 2 minute survey: https://forms.gle/seGETvPsFSfyYeWSA 🙂

github-actions[bot] commented 1 year ago

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

Boostrix commented 1 year ago

what's the status of this, what happened to the poll, any results / outcome ?

github-actions[bot] commented 8 months ago

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

github-actions[bot] commented 7 months ago

This issue was closed automatically because it has been stale for 10 days with no activity.