Dynamic NodeDriver discovery, import resolution and plugins

[jernst]

The need for https://github.com/fediverse-devnet/feditest/pull/178/commits/140eb13521dc79ee52bdd6647a33774390fb100a#diff-0966bbd71ccb431e7bf6dd6a60bc867dac71736ca76a8668d4a1b9f31d26343eL21 seems to be a consequence of the particular way we discover available NodeDrivers.

This issue is basically just a reminder that this kluge exists and is likely a consequence of something (NodeDriver discovery) that perhaps should be fixed instead, if we can find a better way.

In Python, what are some good ways to do "plugins"? The idea is something like:

• the feditest package
• "plugin" packages that give feditest some extra NodeDriver implementations, say feditest-mastodon (which would depend on feditest and mastodon.py)
• and that ... and I don't know how to do that other than with a mechanism such as our current NodeDriver discovery mechanism ... can either be dynamically discovered by fedtest as being installed, or that somehow register with the parent package.

[steve-bate]

I don't know how common this issue will be (where a third party library is the same name as a parent package name). It's arguably a python quirk as much as a plugin-related issue.

That said, a couple ways plugin support is commonly implemented are using setuptools entrypoints (e.g., rdflib) and the pluggy library (e.g., pytest plugins). Both of these techniques allow plugins to be packaged in external libraries (rather than source files in a list of directories). Since those external packages are installed into the python environment, it helps with resolving any library dependencies for the plugin.

For example, the Mastodon support could be implemented in a separate repo (feditest-mastodon) and installed into any feditest environment where a Mastodon node is needed. The mastodon.py library would be automatically installed as a side-effect without a direct dependency from feditest. If any libraries are shared between feditest and feditest-mastodon, those version constraints will be analyzed during installation.