ubiquibot / plugins-wishlist

0 stars 1 forks source link

Personal Agent #3

Open 0x4007 opened 3 months ago

0x4007 commented 3 months ago

This one I'm very excited about. The vision here is that we can make custom user "agents" (i.e. plugins with LLMs) that are hosted by the user's GitHub (so they can modify it) and will automate actions for the user (with their PAT to authorize as them) with the full context of a particular repository/organization.

There are some ways we can make the template code which will be forked:

  1. simple starting point would be just template/boilerplate that doesn't do anything
  2. code makes a call to an LLM (we could even run a small model locally on the GitHub Action runner potentially in order to make dealing with credentials/API keys more hands off, at the tradeoff of it being dumber than ChatGPT etc but decentralization/free is cool)
    1. this LLM has a big prompt in the template that explains the context (you're running in a github action runner and a user invoked you from this repo...) and its capabilities (we can provide some local functions from our SDK that it can invoke to perform specific tasks by using an authenticated octokit instance using the person's PAT. It also receives all of the context of the event invocation (which user called the function, what repository and organization is it coming in from? possibly even scraping all the linked issues and pull requests for more context)
      1. If we can reliably get the LLM to write working code with Octokit (or just raw CURLs with the PAT) then we can have a context aware and english language input to any function a user can perform on GitHub (limited to the PAT permissions) which is quite interesting.
      2. The user can "fine-tune" their LLM by adding extra details and preferences to their prompt in their forked code. I imagine that I would continue to add new sections as I see repetitive questions/queries.

Assuming that the org config enables support for personal agents, technically we can extend personal agent capabilities beyond GitHub. Generic telegram example: @pavlovcik send me the credentials on Telegram @username with the right code in my personal agent, the GitHub Action can send information to their Telegram. All invoked from the GitHub Action runner!

This could make plugin development a lot more exciting and rapid. If the team all works on their own agents, and tests them in production, we could extract useful bits from eachothers' and release "official" plugins which may normally have slower r&d cycles.

In the further future, our kernel can support webhooks coming in from other services (like Telegram) and invoke user agents which can be a very powerful architecture for platform composability. For example, a bot call (can be "inline" in a dm to someone as well) that will pass along the conversation context to our kernel, then to a user's personal agent (github action) back to kernel and then back to Telegram

Notes for @pavlovcik/personal-agent


Comment rewrites:

From my phone sometimes writing comments can be arduous with the custom vocabulary we use and the autocorrect. A simple agent that will save me from a lot of frustration is to edit my comments posted, and correct any typos when I post from my phone.

[^1^]: Although it is not clear to me how we can capture the event from this. I suppose I would need to manually add in the org/repo config for issue_comment.created.