Plugin engine/folder structure

[Mte90]

I think that the quickstart plugin should be like an example plugin with all the parameters that are hardcoded and is disabled by default (used for testing and for documentation like hello-dolly.php for WordPress). Also there should be a plugin for every service OpenAi etc, so like adds endpoints like https://github.com/pieroit/cheshire-cat/blob/main/web/cat/routes/openapi.py (it should be moved in that plugin).

In my opinion cheshire-cat plugins should be installable with pip (support also a repo so not they aren't mandatory on pypi), in this way they are loaded by the app (django style) and doesn't matter the folder. At same time can be handy to have in my opinion and avoid conflicts with git (also if exist .gitignore) to have a folder with the bundled plugins and one "manual".

Let my explain my hybrid example between Django and WordPress for the plugin system I suggest for this project:

Folder management

• Django has a variable that is a list of apps to load in the settings file (in our case can be whatever like a txt file in the root with just the path or the name), automatically loads it from the PYTHON_PATH variable. In this way those can be git folder also private and don't interfer with the program folder that is elsewhere
• WordPress has too a setting with the list of files to load but in our case has another folder mu-plugins where the user can put the plugin manually that are automatically activated and loaded (there isn't the option to disable them in the system)
• Laravel (just to do another example) install the packages by composer so it with another variable you define the classes to load from the packages you want (like Django)

So basically I am suggesting a double solution for folders:

• Package installable with pip that are in a system folder (or virtualenv or docker whatever)
• A folder inside the cat where the user can put plugins that are automatically loaded

With the second solution can experiment but with the first one open the doors to more interesting options.

Plugin engine

So next step is to define how to interface with plugins.
Laravel and Django works that you extend the native classes but this doesn't let you to mix so much multiple classes and is not easy for newcomers/newbie of development.
Another approach is the one by WordPress based on hooks where you can specify callbacks on some keys and the CMS (in that case) check if there is something registered. In this way you can create a queue of code to be executed between various sources. Yes, Laravel offers signals and also Django that are similar but they are forced to a specific set of events on some defined behavior. In our case we want to be more elastic.

Plugin user cases examples

• Before the input arrives to AI the user want to check the content if it is allowed by their policies (like the usual badwords and so on)
• Restrict the content area where the user can ask from AI (like only the document provided for learnings)
• Put a limit of questions per session
• Execute 2 different AI at same time to compare their results
• [...]

This user cases are just example to provide some ideas where in the actua lcode can be interesting to put something that let's the user to extend the behavior.

Resources

• https://github.com/pytest-dev/pluggy
• https://github.com/pieroit/cheshire-cat/blob/main/web/cat/mad_hatter/mad_hatter.py (actual plugin loading engine)
• https://github.com/pieroit/cheshire-cat/blob/main/web/cat/looking_glass/cheshire_cat.py#L47 already present a hook engine but is just a class replacement (not allows to enqueue multiple plugins on the same one)

PS: moved from #63

[pieroit]

I need time to reply to this. Coming back asap

[pieroit]

Plugins are being discussed in other issues, closing this as it has been useful but now too general