merge-plugin doesn't process extra.merge-plugin sections from packages required by the project

I'm trying to create a package (a framework) that, when installed, requires this plugin and configures it to include composer.json files from several directories contained inside that same package. This would allow me to split package requirements between several "modules" inside that package, instead of creating one single, monolithic, big require section on that package's composer.json. Furthermore, I do not want that configuration to be present on the main project's root composer.json, as the user of my package (framework) should not have to know about those internal details, nor pollute his project's composer.json with irrelevant info. All he/she should have to do is to require the package, as usually. The problem is, if I configure this:

"merge-plugin": {
  "include": [
    "private/packages/selenia/framework/subsystems/*/composer.json"
  ],
  "recurse": true,
  "replace": false,
  "merge-extra": true
}

on the project's root composer.json, it works, but if I configure it inside the package (which requires this plugin), it does not. I also tried changing the "include" pattern to be relative to the package's root folder, but to no avail. It seems this plugin only processes extra.merge-plugin sections of composer.json files that are included from the root composer.json. If a package defines its own extra.merge-plugin section, it won't work. Is this so? If it is, could this feature be implemented?

It seems this plugin only processes extra.merge-plugin sections of composer.json files that are included from the root composer.json. If a package defines its own extra.merge-plugin section, it won't work.

This behavior is by design currently. I didn't want random packages to be able to start pulling the plugin into the Composer workflow. Processing "merge-plugin" configuration stanzas only happens in the root package by default. If the root package specifies "recurse": true then configuration found in other merged packages will be processed. There is currently no way to configure the plugin to look for and process "merge-plugin" configuration that is supplied by a package that is installed normally (i.e. a package that is a direct or indirect requirement of the root package).

If it is, could this feature be implemented?

It may be possible, but I don't think I would be comfortable with it being automatic behavior. I would at least want the root package to provide some configuration switch to enable the use of the plugin. With such a switch enabled the plugin would need to intercept and inspect the composer.json manifest of each installed package (or maybe just of a whitelist of packages given in the root package configuration) and process them in the same way that included config files are processed when recurse is enabled.

I see, and I understand your concerns.

Please bear with me a little, as I explain my use case with a little more depth, so that you may see the potential benefits (for everyone, not only for me) of extending the merge-plugin to include the feature I'm requesting.

It would be amazingly useful to be able to composer require my-awesome/php-framework and composer would install the framework and this plugin, and use it to register internal (not installed by composer) packages of that framework, so that their PSR-4 mappings and package dependencies would be automatically installed and configured. This framework I'm working on is using a kind of architecture that is currently gaining steam and more acceptance on the community, which is a monolithic architecture (i.e. the framework's repo contains all the framework's packages) that is still modular, because each internal package is a fully-featured composer package, that can be published on Packagist and installed individually, if one wishes to do so. These packages can be published via a git-subtree split so that they become available as standalone packages.

Here's an article, Monolithic Repositories with PHP and Composer, that explains it further.

This is, in fact, something that is already being done by major frameworks, like Laravel and Symfony,

What I'm doing differently is that my framework's internal packages (or components, as some call them), are using composer-merge-plugin to setup PSR-4 class maps and load their own dependencies (packages), when they are used as integrant part of the single, monolithic framework package. This has many, many advantages. I have all the advantages of modularity while still being able to simplify immensely my workflow by developing and versioning all packages as a single, coherent whole, all together in a single repo.

Initially I used one repo for each package, and published each one on Packagist individually, and managed each package's history and versions individually. It quickly became a maintenance nightmare (we're talking about dozens of packages, here), so I had to find a workable alternative.

composer-merge-plugin is amazing for this. I'm currently able to implement the architecture I described and it works very well. But it requires the framework to be dependent on configurations defined on the developer's project's main composer.json, which:

breaks encapsulation,
pollutes the main project's composer.json with internal framework details that are irrelevant to the app. developer,
creates another weak spot for the introduction of misconfiguration problems (as the app. developer can mess it up),
makes it more difficult to update merge-plugin configurations as the framework evolves (as they are outside the framework's scope and reach).

It seems to me that, if a package wants to have its own subpackages managed be merge-plugin, that is an internal implementation detail that should not interfere with the package's host project or any other packages being used by it, nor should it bother the developer (making him/her have to mess with the main composer.json), nor should it present or cause any kind of problem that should bother the developer.

To solve this, I believe that a package should be able to have its own merge-plugin configuration that would not interfere in any way with the main project's merge-plugin configuration (if any) and would be limited to:

include composer.jsons that lie inside that same package's directory tree (they configure code that it's internal to the package, so why not?)
have those internal composer.jsons register stuff in the autoloader, merge 'extra' config sections and require additional packages. (I know one could merge all those internal composer.jsons into the package's composer.json, which would do precisely the same thing, of course, but that would hinder development and make it less modular).
not interfere with other package's merge-plugin configurations, including the main application's. Each package requiring the merge-plugin would have its own isolated configuration universe.

Yes, the developer may also be able to disable this behavior by using some configuration switch, I'm not opposed to that, but is it really necessary? If a developer does not want a package to install and/or use the merge plugin internally, then he/her has no recourse but to not install that package at all, as it would not work properly without that functionality, anyway.

I hope I have given you some food for thought. I'd like to know your opinion on this. If you decide not to implement this, I will have to require my framework's users (developers) to copy paste some weird stuff to their app's composer.json, and repeat that, as necessary, as bugs and further functionality are added to the framework. It's not the end of the world, of course, but in a world where "monolithic composer-based development" is gaining traction, having this plugin support it fully would make it more useful, now and on the future. Who knows if some major framework might even start to use it internally, at some point? ;-) Regards

@claudio-silva Getting the contents of all of the package definitions in time to take action on them would be the main thing to be concerned with. If I'm understanding your scenario correctly, the definitions that you want to merge won't exist on the host where they are being installed until after Composer fetches and installs some other package. At that point it would be too late to adjust what Composer is going to do as the solver would have already run to determine all of the needed changes.

I'm not sure that this is really solvable in any reasonable way even with changes directly in Composer without a thorough rethinking of how Composer finds and resolves the package dependency chain. The closest thing you could do would be to cheat like composer-merge-plugin does to bootstrap itself and fire off a second round of Composer running on initial install of any package that declares includes. And you'd have to repeat that until there were no more includes that showed up. Doing this could cause really strange things to happen in the most perverse case as each new include could introduce a conflict, upgrade or downgrade with a dependency that had already been chosen via other packages.

If you want to play with it and see if you can come up with a patch and working tests I'd be willing to review it.

@claudio-silva Have you seen the documentation on "path" repositories? I just stumbled on it while researching another issue and realized it may handle most of your monolithic repo issues.

wikimedia / composer-merge-plugin

merge-plugin doesn't process extra.merge-plugin sections from packages required by the project #77