V2 Async Modular Architecture

[Swiftyos]

Description

This issue serves as a placeholder for the implementation of the v1 async modular architecture, which includes the introduction of AICore with abstract classes, potential server integration, and an expanded plugin system. The goal is to restructure the code base to make it more maintainable and scalable by leveraging asynchronous agents, a modular design, and an enhanced plugin system.

We encourage everyone to participate in the discussion and link their related PRs that contribute to the implementation of this architecture.

Tasks

• [ ] Design and document the new async modular architecture possible including an async AICore, server integration, and expanded plugin system.

Please feel free to join the discussion and link any PRs that you feel are related to this issue. Your contributions and insights are greatly appreciated.

Current contributions to this approach:

449 #757 #765 #1937 #2442

[ntindle]

Please see my comments on #757

[dwrtz]

for async server see #449

it's not stateless but gives an http api that can serve simultaneous sessions

[ntindle]

TLDR on the thread, require manifests for plugins as close to the start of the project as you can.

Also have a good number of sensible plugins built in by default but disabled so that the repo that gets the most love is this one

[arrenv]

It is possible my PR https://github.com/Significant-Gravitas/Auto-GPT/pull/1937 may align with this in terms of making agents and their output more structured, would be interested in talking about it some more. I'm currently scaling back my scope so that it has a better chance of being considered for merge.

[Swiftyos]

Thank you @arrenv @ntindle and @dwrtz. Your PR's are noted:

449

757

1937

This is a major step forward for Auto-GPT and many people are making steps towards its. This discussion is to capture how we collectively what we feel is the best way to architecture the system

[Wladastic]

been dealing with lot's of async/sync issues with the telegram bot server I wrote. I experiment with using this command: os.system("python -m autogpt {}".format(" ".join(sys.argv[1:]))) would be super great to have autogpt be async, as it took me a whole week to get the message consumers to run and not block each other or autogpt's async threads.

[ntindle]

Re: the expanded plugin system ft. some thoughts combined from another thread

I think you should take some pointers from the Home Assistant project. Their integrations methodology has successfully allowed generally useful commands while providing ways to install less common integrations like those described here.

General Diagrams: https://developers.home-assistant.io/docs/architecture_components

Manifest files: https://developers.home-assistant.io/docs/creating_integration_manifest

https://developers.home-assistant.io/docs/dev_101_services

Integrations in Home Assistant are like available capabilities. Out of the box, only some of them are enabled. As you add devices, you will allow integrations to use them. Each of these integrations has its rules for helping, whether requiring an API key or accepting terms and conditions.

Internally, all the logic for these is in their own Python packages, and each "integration" in the core Home Assistant repo wraps these up tidily.

I would like to propose the same here. Each "integration," such as Twilio, Telegram, or Wikipedia, would live in its directory within autogpt/commands. Within that directory would be a manifest listing the required integrations for it to work, the commands it exposes, and the dependencies that must load before it does. Within the actual code for the integration, a configuration and setup entry point would be defined, as well as all the commands. Each command would point to a Python package (as reasonable and required).

A config tool (ideally UI, but a config file also works) would allow users to specify the list of integrations to load (with a sensible default also enabled). This would also be easy to expand as future changes that are being discussed are rolled in.

This would clear up the problem of loading questionable code into an automatic execution engine while still allowing intelligent users to do it.

I prefer a plugin system that doesn't require code changes for the user to swap plugins. It makes sense for many systems, but there's a high likelihood this system will get a UI in the future, and it will have fewer and fewer developers as the primary users. As such, most of this process must be automatic and not painful for the average individual. A manifest file is practical for automated processing and loading. It's also a convenient place to track info like descriptions for UIs later on.

Additionally, with a manifest, an awesome list wouldn't be required. A repo could hold a reference, and all plugins be loaded via a UI/CLI similar to most package managers.

I propose (imagine this is perfectly valid yaml syntax, not typed on my phone yaml syntax):

manifest_version: 1
plugin: twitter
name: 'Twitter Plugin'
description: 'Connect AutoGPT to Twitter!'
capabilities:
  - command: send_tweet
    description: Send a Tweet
    aliases: 
      - tweet
      - create_tweet
load_after:
  - http
requirements: pytwitter

Imagine all those values are valid and meaningful. That could be helpful long term.

I'd also consider a field for code owners to help some of the change management processes that have gotten hard to manage recently. The CI could parse this and automatically assign code reviewers and labels to specific projects and categories.

codeowners: [ @ntindle, @TaylorBeeston ]

I'd see this project becoming the tool developers want their stuff in, not the tool developers use to build their products. The former makes a strong community, and the latter fragments it. It's a significantly more sustainable model from a funding and branding perspective. Take a look at Babel and Core-JS. One has bills paid by corp donors, and the other needs help with branding and paying bills constantly.

Putting all these (admittedly complex) tools into place now is an excellent way to get there.

[zachary-kaelan]

@ntindle I'm surprised you didn't mention OpenAI's ChatGPT plugins, which do in fact just consist of a JSON manifest on the API's domain and a YAML specification documenting all the endpoints.

Optimally AutoGPT should itself be able to select and install the plugins it thinks it needs, with user confirmation.

[Swiftyos]

@collijk Do you want to link this to what your doing or just close this ticket?

[github-actions[bot]]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions[bot]]

This issue was closed automatically because it has been stale for 10 days with no activity.