Plugin system

[FichteFoll]

this is a changing post with our todo list

Plugins should:

• [x] be able to hook into pretty much every event or functionality so that the core can remain as slim as possible
• [x] use asyncio
• [x] have dependencies that we resolve
• [x] be separated in namespaces (and systems) for core and user plugins
• [ ] be able to specify conflicting plugins, in which case the plugin isn't load and an error emitted

For the future:

• [ ] be reloadable, thus have unload and reload handlers (or one handler with a param)
• [ ] auto-reload if changed on-disk. Reloads of dependencies need to cascade (unload dependant plugins first, recursively)

Questions:

• [x] Where should plugins be sourced from? Specific directory or user configurable? Multiple locations, e.g. "core plugin" and "user plugin"?

[nitori]

Some things you might want to add:

Plugins should also be able to add ircv3 capabilities (CAP), and the treatment of those plugins might be different. There is no reason to load a plugin with a specific CAP, if the server doesn't support it.

Also as a question: what would happen if a plugin provides more than one CAP? Should that be allowed? And If it is, how to separate the capabilities that are supported by the server from those that aren't?

The easiest might be to just not allow it, or maybe to bind hooks or similar to a specific CAP, e.g. "only register this hook to the network if it supports the CAP".

Pseudo: register_hook('connect', on_connect, cap='sasl')

[nitori]

A few tools I found:

pluginbase @ pocoo

pluginloader

pluginmanager

None of these seem to have any dependency loading or unloading/reloading mechanics. So we either need to look more or see if one of them is easily extensible and fit for our purpose. Note: I don't particularly like any of them.

[nitori]

In order to understand what our plugin engine should be able to do, we have to consider what typical (or non-typical) bot functions you usually see on IRC.

A list of the things I came up with from the top of my head. I use the ! prefix here even though the "Shanghai" bot officially uses the plus sign (+):

• small stupid play-commands, e.g. typing !foo yields a simple reaction. Those are easy to implement.
  • though cases for persistence of data are probably necessary. Typing !cookie might give a user the ability to occasionally (time-out based) collect cookies or maybe something like small memos a user can keep.
• Stuff like !seen command, or reminders (remind me when user X comes on, or notify user X when he/she comes on).
• Possibility to add aliases to commands, !g in addition to !google.
• Timed commands. The current Shanghai is able to remind users about things in a given time period, using the !timer command.
• Timed events. Things that happens externally, either by notifying the bot in some way (github webhooks?) or by making the bot request updates regularly (RSS/Atom Feeds).
• Fetching data from resources. Shanghai looks up URLs posted in a channel and outputs metadata about it's contents, e.g. file type/size, webpage title, image formats, even video title/formats (matroska only currently though, using enzyme which is not asyncio compatible).
• The already mentioned ability for adding capabilities (CAP command), which possibly also implies the ability to extend the protocol itself.

[FichteFoll]

re CAPs: I'm thinking of having a "request_capabilities" hook where plugins get handed a set of the provided capabilities by the network and can return a set of those they want to request. The plugin manager joins the sets and sends them off as a request to the server. Supporting multiple CAPs wouldn't be an issue that way.

There should probably be an attribute for the available and active capabilities for each connection so that plugins can access that. Filtering a hook to only apply if a certain CAP is active should be trivial then. We could provide some of these more advanced hooks as a plugin or add everything to core, idk.

Also note that there is a CAP that allows updating the network's available CAPs (mostly useful for bouncers I suppose).

[FichteFoll]

I wrote down my general idea of plugins in our wiki: https://github.com/chireiden/shanghai/wiki/Plugins

Please review and comment or ask questions.

I put hot-reloading and depending on pypi packages on-hold as they are not required for the basic concept.

[nitori]

Regarding Hook priority: I'd think that a plugin that depends on another plugin might want to ensure that certain hooks are always executed after (or maybe even before) the same hooks of the other plugin, without knowing the specific hook priority of the plugin it depends on.

E.g. plugin A depends on plugin B, and plugin B hooks into, let's say, privmsg events. However, plugin A wants to either modify B's output (ensuring the hook in A is called after B) or prepare it (ensuring the hook in A is called before B).

Simply using priorities might not be sufficient. If the developer of plugin B decides to change the priorities, plugin A has to be changed as well.

I have no idea how to do that though, maybe every registered hook can be accessed via a name? Plugin B plugin key might be "pluginb" and it registered the hook "privmsg". So plugin A might be able to say "please register my privmsg hook after pluginb.privmsg" or something like that.

Pseudo: plugin_system.register('privmsg', after=['pluginb.privmsg']) or the like (a list because it might be more than one).

[FichteFoll]

That would make things complex for a couple reasons.

1. We would need to manage a two-sided (?) dependency resolution for executing hooks before or after certain other hooks and we have no idea what to do if the provided dependencies collide (i.e. A before B, B before A)
2. Plugin authors most likley do not (and usually should not) know which other plugins are currently loaded and what they do outside of their dependencies. And these dependencies are module-level. Depending on the exact action that A and B want to do, it might b better/easier to just monkey-patch the A plugin from B's code or for A to provide an API for these situations. The priority level system is specifically designed to prevent plugins from the requirement of knowing what other hooks are currently active.

[FichteFoll]

33 and #34 implemented most of this.

What remains is unload- and reloadability of plugins, auto-reloading when changed on disk (low priority) and all the config stuff.