search

spongedsc / pathways

Pathways: multi-modal AI/ML models on discord
GNU Affero General Public License v3.0
3 stars 1 forks source link

[MERGED] RFC: Callsystems #77

Closed daniwasonline closed 4 months ago

daniwasonline commented 4 months ago

target: rfc, unknown

Important terms

Description

In 1.x and 2.x, Pathways has one singular backend. While this backend is relatively robust, it only supports two contextual functions outside of the model itself: Imagine, the image generator, and image recognition. This is very limited. We aim to address that limitation with #74, the Integrations RFC. However, for us to be able to develop Integrations, we need to have a "flexibly stable" system that we can build on top of. This is the goal of the Callsystems RFC.

What is a callsystem?

The callsystem is a fancy term for the backend system that powers passive activation. On a messageCreate event, Core takes the message and checks to see if it meets the criterion for a passive activation. If it does, Core instantiates an instance of the callsystem and activates it. From here, the callsystem manages the bulk of the process (from fetching context, to calling the model, to sending the response). The callsystem passes control back to Core with information about how the task went, which can be used for debugging purposes.

How will callsystems be loaded?

In 2.x, the bot already performs a sweep of pre-defined structures on startup. This includes slash commands, actions (context menu interactions), and events. The bot will load callsystems in a similar manner (the changes required for this have already been introduced in #76). After the sweep, the callsystems (which are versioned!) are loaded into a Map and injected into the client context. During messageCreate events, the configured callsystem is pulled from the Map and the callsystem class is instantiated.

Why do we need a standard API for the callsystem?

While building out a full standard for a separate callsystem may sound much more tedious than simply keeping all the logic within Core, having an established interface and a standard library of helper methods improves the experience when building out a callsystem. This is especially important when building a system as modular and sophisticated as the standard proposed in the Integrations RFC, which further abstracts the callsystem into third-party-compatible integrations.

Having a standardised interface also allows for the callsystem to be changed on-the-fly, similar to how the instruction set can be changed without a restart.

What will happen to the old backend/callsystem if the RFC is approved?

The old callsystem has already been partially integrated under the new Callsystem API as part of this RFC's implementation and will be referred to as Legacy going forward. In future commits, we'll be further integrating Legacy by migrating certain redundant APIs to their respective standard library replacements, in addition to having it return the proper CallsystemActivationResponse for debug.

If/when this RFC reaches main and stability (see Target), Legacy will briefly remain the sole callsystem until a potential Integrations RFC reaches main in a following minor bump. UPDATE: We are now targeting 3.0 for Integrations.

In the long term, we don't expect to leave Legacy unmaintained. While it won't ever be fully integrated as a first-class citizen of the new standard (or, at least, not for a long while), we still intend to maintain it and will potentially try to backport a subset of future improvements & features to it. Integrations will likely also use Legacy as a fallback in cases where it fails.

Implementation

An RFC implementation (stable proof of concept) can be found in the rfc/callsystems branch (part of #78). It is likely that this implementation is final and will not have major rebases/refactors going forward.

Standards reference

The standards introduced in this standard can be found in doc/standards in the rfc/callsystems branch.

Target

The implementation of this RFC would result in massive breaking changes. As such, we are targeting the next major bump (i.e. 3.0) for this change.

artifishvr commented 4 months ago

The only question this leaves me with is that the "passive activation" would look something like throwing messages at something like firefunction, to decide which "path" it would go through, right?

artifishvr commented 4 months ago

also this is supposed to be one of your hobby projects rightttt?????? this is so in-depth togif

artifishvr commented 4 months ago

when do we move to BSL and make this SaaS and pay you

daniwasonline commented 4 months ago

The only question this leaves me with is that the "passive activation" would look something like throwing messages at something like firefunction, to decide which "path" it would go through, right?

this API is supposed to be the base for a system that handles what you said (i.e. #74 will be built on top of this API). analogy because i'm actually too dumb to explain it using proper terms in a simple way:

daniwasonline commented 4 months ago

The only question this leaves me with is that the "passive activation" would look something like throwing messages at something like firefunction, to decide which "path" it would go through, right?

this API is supposed to be the base for a system that handles what you said (i.e. #74 will be built on top of this API). analogy because i'm actually too dumb to explain it using proper terms in a simple way:

  • imagine a dinner table (the channel that it's listening to). on this dinner table, there are multiple plates (messages).
  • sometimes, a person (the bot/core itself) will pick a plate from this table (a message that meets the requirements for an "activation").
  • now, the person eats the food on the plate. say the food that this person just ate had a raw egg (the currently configured callsystem), and a baby chick hatches inside of the person's stomach. (this is where the callsystem takes the message and adds contexts, calls functions, then calls the model and prepares a message response)
  • a few minutes later (obviously not that long in context of our stuff), this person feels sick and vomits all over the table. (the "vomit" is a message response with content/files/embeds/whatever)

that analogy is rancid as fuck but it perfectly summarises what a callsystem is doing here. in our analogy, the integrations callsystem (#74) would be the special raw egg with a birthing chick

daniwasonline commented 4 months ago

also this is supposed to be one of your hobby projects rightttt?????? this is so in-depth

people who've known me long enough will know that i am extremely thorough in projects that i'm passionate about

(you should see the git repo for newssmart, i was the only one working on it and i had detailed asf rfcs)

daniwasonline commented 4 months ago

when do we move to BSL and make this SaaS and pay you

2028, profit-splitting system 50/50

kidding, i do this shit for free and i'm gonna keep going as long as i'm motivated

daniwasonline commented 4 months ago

API Freeze

At this point in time, Callsystems and libraries/components related to it (i.e. CallsystemStd and CUTS) have reached API stability and are now suitable for a full release. As such, Callsystems will now enter a full API freeze. This means that no breaking changes or spec tweaks will be made from now until the RFC implementation is merged into master (est. premajor 3.0.0), with the only exception being bug fixes and security patches to the implementation itself.

Issues and PRs regarding this RFC will not be closed and new ones may be submitted during the API freeze, but only those which meet the bug/security patch criterion will be accepted and/or addressed. Breaking changes, spec tweaks, and general quality of life improvements will be on hold until the API freeze is lifted.

daniwasonline commented 4 months ago

RFC Implemented

The Callsystems RFC implementation in #78 has been merged into main. This RFC will remain open for comment for 14 days, after which a new RFC will be required to propose any significant changes to the Callsystems standard.

The GitHub issue will remain closed to keep things tidy.

daniwasonline commented 2 months ago

RFC "Open For Comment" window closed

Callsystems's post-merge comment window concluded on the 13th of June, 2024. Significant changes to the Callsystems standard will require a new RFC going forward.