Plugins: support "require" other plugins

[mattab]

In #4183 support is added for "require" piwik and / or a php version.

• Some plugins may also "require" other Piwik plugins (core or third party) to be installed at a particular minimum version.
• When a plugin has other required plugins, when possible, ask user & then install required plugins first, then this plugin.

As a result, it will be clean and almost error-free to use a plugin, as long as the developer set the "require" flags properly. This would be a subtle yet useful new feature.

[hpvd]

yes this is looking very very useful to me too. especially because I'm (like most of the piwik users) no programmer and that's why I like if things work as aspected "out of the box" - without having a great interesting challenge before working out how it might work... :-)

=> maybe we should implement this as early as possible (2.1?) because this is one of the great features which guarante e a hassle-free user experience for everyone which may prevent users talking about piwik as a "tinkering software" which still can only be used (for advanced jobs) by skilled programmers... which may influence other adaptors negatively in their decision turning to piwik.

btw: do you see more of this features helping for a hassle-free user experience?

[mattab]

Some work was done by @tsteur in #4183

[mattab]

I must have closed it by mistake. This feature is actually very useful to let plugin manager be aware of plugin dependencies and erroring when trying to install a plugin with missing requirements.

[hpvd]

yes -"erroring when trying to install a plugin with missing requirements." and also the other way around? when de-installing/deactivating a plugin which is needed by an other?

[mattab]

The goal is that when we activate a plugin via the UI it will ask to first activate all other plugins (and lists here also the list of dependencies of those plugins recursively if any).

It will also be practical to use with ./console core:plugin activate PluginX whihc will activate pluginX and all other depedencies too.

In MVP we can simply error if the required plugins are missing from plugins/ directory. In a later version we could ask users to download them from marketplace if they are there.

[tsteur]

I haven't tested it today but it might be already implemented as MVP: https://github.com/piwik/piwik/blob/master/core/Plugin/Dependency.php#L81

and +1 as suggested by @hpvd to later also check on deactivate etc.

[tsteur]

FYI: When trying to upload a plugin that requires another plugin I do already get the following error: "Please update Transitions 2.9.0 to a newer version, Transitions >=5.6.0 is required." or if plugin is not installed "Please update Transitionsd to a newer version, Transitionsd >=5.6.0 is required." (Error message should be like "Please install Transitionsd >= 5.6.0 as it is required by the XYZ plugin"). I presume the same is happening for Marketplace

[tsteur]

Just had a look and it is done whenever a plugin is installed or updated via Marketplace or Zip-Upload.

Not sure if still needed to check on activate for MVP as such a plugin cannot be installed in theory (unless someone copies it manually into plugins directory which is maybe not recommended? Rather we maybe offer a command to install a plugin from a zip file or so?) .

As said earlier :+1: for also checking on deactivating any plugin in case another plugin needs it etc

[mattab]

Not sure if still needed to check on activate for MVP

we need to check on activate, because putting plugins in the filesystem is still one of the documented ways to install a plugin

[tsteur]

See comment above. Maybe it makes sense at some point to no longer recommend it (remove it from documentation) and replace it with a console command or whatsoever. Could maybe make sense long term to let all plugin installations go over one code path in the end to have less bugs, less code, less complexity etc.

[mnapoli]

Copying the links I shared on Slack here:

• https://github.com/wikimedia/composer-merge-plugin
• http://www.mediawiki.org/wiki/Composer/For_extensions

[mnapoli]

Summary about the idea of using Composer for installing plugins so far:

• using Composer would provide the following benefits:
  • plugins could require other plugins
  • plugins could require 3rd party libraries
  • users could install plugins using Composer
• we could have 2 files:
  • composer.json (same as now, committed and distributed)
  • composer.local.json (optional, not committed like config.ini.php) containing plugin requirements
  • Thanks to wikimedia/composer-merge-plugin those 2 files would be merged (in memory) and used by Composer to install dependencies
• plugins would be installed by running composer install (or update?)
• the marketplace could install plugins by adding them to composer.local.json and running Composer
• Composer would be embedded in Piwik (we can require composer/composer to include it)
• 3rd party libraries would be installed to vendor/ (like today) and plugins would be automatically installed to plugins/ by using the piwik-plugin type defined here
• plugins that are in submodules today could be installed by putting them in require-dev, and I guess by requiring dev-master that would clone master and let us edit the code/push just like today (hopefully)

To avoid our marketplace to be dependent of http://packagist.org/ and to protect the privacy of users, we could setup our own Satis (open-source) or Toran Proxy (commercial). That would also allow us to put our private repositories there, like for example the Piwik Pro plugins.

For Piwik installs that have no connection to internet, the situation would still be the same:

• plugins can be installed manually by uploading inside plugins/
• we could also provide a service where people can upload a composer.local.json (for example) and our website would build a zip containing Piwik completely setup with the plugins (with the correct versions). That way to update Piwik (and the plugins) people would just have to upload composer.local.json and replace Piwik files with the archive they downloaded… That's just a random idea, it's a big piece of work I guess…
• they could also maybe run composer install locally and then upload the result. This solution is more complex, but anyway manually uploading plugins would still work so no biggies.

This is just a summary to keep track of ideas, nothing has been thought through from end to end. If you have remarks please share.

[mnapoli]

Related: https://github.com/mybuilder/conductor introduced here http://fr.slideshare.net/StenHiedel/symfony-uk-meetup-21-may

[tsteur]

I'm not sure if mentioned here but if there is an update for a plugin that requires another plugin or if there is an update for a plugin that is required by another plugin we need to make sure to ideally update both at the same time in one go or to perform a check upfront whether afterwards both plugin still will work with each other. If they aren't compatible with each other we should inform the user upfront about this problem.

Example: Plugin CustomDimensions is required by Plugin CustomReportBasedOnCustomDimension.

If one updates first CustomDimensions then there is a (rather small) chance it will break plugin CustomReportBasedOnCustomDimension and the platform would automatically disable this plugin under circumstances without informing the user (eg if require information of plugin CustomReportBasedOnCustomDimension is no longer met).

If one updates first the plugin CustomReportBasedOnCustomDimension then there is a good chance that the new version doesn't work with the still older version of CustomDimensions as it possibly requires a newer version of CustomDimensions. In this case the plugin CustomReportBasedOnCustomDimension would be disabled by the platform without informing the user etc. Problem is once someone installs the required CustomDimensions version in the next step we would not automatically activate the plugin CustomReportBasedOnCustomDimension again. For a user it will be very hard to notice that the plugin was disabled by the system and that it can be enabled again somewhere in Piwik (and one has to know the plugin name etc).

[tsteur]

FYI: On the Marketplace we will soon check whether a required plugin actually exists. This means if plugin FooBar requires FooBaz we will check whether FooBaz is either a core plugin or a plugin on the Marketplace. If it is not we will throw an error and inform the developer about the unknown plugin FooBaz via email and won't create/update this plugin.

Once we work on this feature #4485 we should update the Marketplace as well to check for the version constraints which can be quite complex since one could specify version dependencies like this >=0.2.0 | 0.3.0 | !=4.0.0. This means if a plugin on the Marketplace is about to be created or updated we should make sure that the required plugin version constraint can be actually met.

[tsteur]

FYI: I spent about 90 minutes looking into Composer and worked on a proof of concept. Problem: There is basically no documentation about using composer as a library (apart from outdates PHP doc online but the classes etc are not documented anyway). Next problem: The classes are quite big and lots of code is hidden in big private methods, many of them untested. However: I believe I understood how it works and how we could use it to resolve the dependency graph see https://github.com/piwik/piwik/tree/4485_composer or better https://github.com/piwik/piwik/compare/4485_composer#diff-41646e8df310953f164f51bb15036224R41 I haven't tested a single line but I think it makes sense like this and it could work. We'd just need to create a couple of good tests as composer is basically not tested and we cannot trust it. Maybe we could work on this during Piwik 3.0