[Proposal] Support for structured python plugins

Context

I'm a sucker for well-structured projects, especially as they become more complex.

When writing albert-translate, I wanted to keep the __init__.py as minimal as possible so that the actual plugin code would be separate from the metadata. The idea was that __init__.py would simply serve as a bootstrap for the plugin and its initialize and handleQuery events.

Unfortunately that couldn't be achieved because in my tests, Albert required all Python code to be in a single file, so I compromised by storing __init__.py and a supporting json file in src/ while icons, build files and other supporting resources/documentation are all structured separately in the root.

Proposal

While supporting a src/ dir is relatively simple, it's just one part of the proposal. The others parts are more involved, so I'll detail each separately. This is mostly an idea at this stage and some parts (esp. config) could depend on how other plugins workaround it currently. Please feel free to discuss any concerns/suggestions.

1. Support for nesting `init.py` with a top-level `src/` directory

Currently albert will detect plugins based on the following paths:

modules/[pluginName]/__init__.py
modules/[pluginName].py

After some digging, it turns out we can add a third modules/[pluginName]/src/__init__.py option with very little effort and without affecting the structure of existing "core" plugins. I've made a promising start on a fix for that locally so I'll create a PR soon.

2. Support for plugins made up of multiple .py files

The second part is probably a bit more involved. Ideally __init__.py should serve only as the main entrypoint, with classes and groups of related supporting functions all (optionally) saved separately in their own files.

Using albert-translate as an example, at least 50-60% of the code in __init__.py is all helper functions that deal with config, arg parsing, making items and loading/parsing language codes. Those could be abstracted into 3 or 4 dedicated files, and after a small refactor of initialize()+handleQuery() the length of init could end up being around 50 lines vs 228 currently.

If I can find time I'll whip up a sample structure, but there are probably at least a couple core python plugins that could be made easier to maintain by refactoring parts into separate files.

3. Addition of config helpers for plugins to utilise

Could be useful here to gather a list of any plugins in the wild that rely on configuration of some sort. Currently mtr stores its config in ~/.config/albert/MultiTranslate/language_config.json while albert-translate uses ~/.config/albert/translate.ini.

Since config files are handled manually by each plugin, there's no guarantee that a plugin's config will be where you expect it. If we were to scan ~/.config/albert/plugins for [pluginName].ini files similar to how plugin loading works, we could handle loading in a single place and provide a config object to the plugin so that it can get/set values with no extra boilerplate.

By explicitly searching only in the plugins/ dir, we avoid any conflicts with existing plugins. The other benefit of handling plugin configs automatically is there's a much greater chance that all configs will use the same format, making it easier for new users to edit them.

All in all, I think collectively these ideas would make it easier to develop, debug and maintain longer / more complex plugins, providing a clear and easy way to structure plugins without ending up in indent hell with a long string of ifs in the handleQuery() method. If we're moving toward submodules and recommending third-party plugins are all stored in their own repositories, there are likely to be more cases where confining actual plugin code within src/ will prove useful too. I'd be interested to hear others' thoughts, hopefully it's not just my OCD tendencies!

I've made a promising start

what do you have in mind? as long it is a python module from the outside you are free to do what you want in your repo.

Just to be sure, have you read the docs on python extensions?

Addition of config helpers for plugins to utilize

Fortunately we discuss things. I found out that a lot of my ideas fell by the wayside. The original idea of these

Function	Description
`cacheLocation()`	Returns the writable cache location of the app. (E.g. $HOME/.cache/albert/ on Linux)
`configLocation()`	Returns the writable config location of the app. (E.g. $HOME/.config/albert/ on Linux)
`dataLocation()`	Returns the writable data location of the app. (E.g. $HOME/.local/share/albert/ on Linux)

was to give the user a predefined location for each purpose. The id of the plugin should be appended. however the core does not know who is calling the function. so we have to rely on plugin authors adhere to conventions. obv they are not written down yet, at least not in a central and generic way. Considering paths most plugins do de following :

Cache goes to <cacheLocation>/<pluginId>/.
Data goes to <dataLocation>/<pluginId>/.
Config goes to <configLocation>/<pluginId>/. You could also directly use <configLocation>/<pluginId>.conf Native plugin config goes to the app config, because they can use QSettings. (thats another story. I don't like the ini format and its limitations. Once I wanted to implement a JSON config reader, but well time is an issue. and I though Qt will come up with one on their own). Also maybe exposing Qsettings to the Python API makes sense for the sake of consistency and integrity.

I've made a promising start

what do you have in mind?

Nothing drastic. I'll add a PR later, already have two different implementations (one only two lines) but after 3 days on this and albert-translate I need a break.

as long it is a python module from the outside you are free to do what you want in your repo.

For the most part everything works great, but I've yet to have any luck importing additional "local" modules. Take the following example structure (where the lang module contains the class Lang):

someplugin
├── __init__.py
└── lang
    └── __init__.py

If you try to import lang it throws a ModuleNotFoundError with "No module named 'lang'".
Trying from . import lang gets "No module named 'albert'".

I've tried other variations like from lang import Lang etc, and even moving lang/init.py to the root as lang.py, but all to no avail. I could just be missing a magic module name to get the import working, but not sure. If you know the right combination to make it work, I'd love to hear it.

Just to be sure, have you read the docs on python extensions?

Of course. Many times in fact. I actually use the configLocation() function in albert-translate already.

I started writing this response hours ago then fell asleep after dinner. Since waking up and checking emails, my mood isn't the same, so it's better I come back to it with a fresh mind and respond to the rest later.

Dave, as always, RL first. I don't expect you to reply fast. Take a break whenever you want. Thank you for your efforts.

Albert imports the plugins in the albert namespace. I dont know which implications this has. I have never coded any huge modules. Maybe adding albert somewhere in the import statement helps.

Support for plugins made up of multiple .py files

This can be done without any modifications to the python plugin or albert core. All you need to do is to add

sys.path.append(os.path.dirname(__file__))

To your main __init__.py module and you can import any other files from your plugin. It may clash with other plugins having the same module files, but simple namespacing should fix it.

Of course treat it as a workaround.

albertlauncher / albert