search

wikimedia / composer-merge-plugin

Merge one or more additional composer.json files at Composer runtime
MIT License
935 stars 160 forks source link

Merging files installed to vendor by other packages does not work on initial run #165

Open schmunk42 opened 6 years ago

schmunk42 commented 6 years ago

I am playing around with a setup where I want to include (merge) composer.assets.json files which reside in vendor and are therefore not available on the first run.

My composer.json.

First run...

composer update -vv
    1/4:    http://packagist.org/p/provider-latest$b3160f77308b723aeb638cf56dac6f48f1ec9d91cf4aeca4e5ec93d496292aa5.json
    2/4:    http://packagist.org/p/provider-2017-07$c6c63e6fc54ee7337b1c55dfc623b61eb3da483c06a624bd317e97b98b433533.json
    3/4:    http://packagist.org/p/provider-2015$a1fedc3267d58ba9f4377eec5b7e69bb19711787f86cbff939e9eab80b247152.json
    4/4:    http://packagist.org/p/provider-2018-01$31fbf249873f6ab209c38b30cbe8851d4a4dd4d708e5503c05a749d226ce392c.json
    Finished: success: 4, skipped: 0, failure: 0, total: 4
Loading composer repositories with package information
[...]
Updating dependencies (including require-dev)
Dependency resolution completed in 0.000 seconds
Analyzed 531 packages to resolve dependencies
Analyzed 172 rules to resolve dependencies
Package operations: 9 installs, 0 updates, 0 removals
Installs: yiisoft/yii2-composer:dev-master 58382e6, wikimedia/composer-merge-plugin:v1.4.1, psr/simple-cache:dev-master 408d5ea, psr/log:dev-master 4ebe3a8, psr/http-message:dev-master f6561bf, cebe/markdown:1.1.2, yiisoft/yii2-dev:2.1.x-dev 3e25001, yiisoft/yii2-jquery:dev-feature/nam c2bf8a8, codemix/yii2-streamlog:1.2.1
  - Installing yiisoft/yii2-composer (dev-master 58382e6): Cloning 58382e62e592b7cd800ab1e9f4a3cf54f3663cce from cache
  - Installing wikimedia/composer-merge-plugin (v1.4.1): Loading from cache
 Extracting archive    REASON: Required by the root package: Install command rule (install wikimedia/composer-merge-plugin v1.4.0|install wikimedia/composer-merge-plugin v1.4.1)

  [merge-plugin] composer-merge-plugin installed
  - Installing psr/simple-cache (dev-master 408d5ea): Cloning 408d5eafb83c57f6365a3ca330ff23aa4a5fa39b from cache
  - Installing psr/log (dev-master 4ebe3a8): Cloning 4ebe3a8bf773a19edfe0a84b6585ba3d401b724d from cache
  - Installing psr/http-message (dev-master f6561bf): Cloning f6561bf28d520154e4b0ec72be95418abe6d9363 from cache
  - Installing cebe/markdown (1.1.2): Loading from cache
 Extracting archive    REASON: yiisoft/yii2-dev 2.1.x-dev requires cebe/markdown ~1.0.0 | ~1.1.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.0-rc, 1.0.1, 1.0.2, 1.0.3, 1.0.x-dev, 1.1.0, 1.1.1, 1.1.2].

  - Installing yiisoft/yii2-dev (2.1.x-dev 3e25001): Cloning 3e25001a6757c8c65dabb174b3d008a266cce544 from cache
    REASON: Required by the root package: Install command rule (install yiisoft/yii2-dev 2.1.x-dev)

  - Installing yiisoft/yii2-jquery (dev-feature/nam c2bf8a8): Cloning c2bf8a85f00061a3728403bac743de66c274f5b4 from cache
    REASON: Required by the root package: Install command rule (install yiisoft/yii2-jquery dev-feature/nam)

  - Installing codemix/yii2-streamlog (1.2.1): Loading from cache
 Extracting archive    REASON: Required by the root package: Install command rule (install codemix/yii2-streamlog 1.2.0|install codemix/yii2-streamlog 1.2.1)

yiisoft/yii2-dev suggests installing yiisoft/yii2-coding-standards (you can use this package to check for code style issues when contributing to yii)
yiisoft/yii2-dev suggests installing ezyang/htmlpurifier (required at `yii\helpers\HtmlPurifier` for 'html' data format support (e.g. `yii\i18n\Formatter:asHtml()`))
Writing lock file
Generating autoload files
  [merge-plugin] Loading vendor/yiisoft/yii2-jquery/composer.assets.json...
  [merge-plugin] Merging npm-asset/jquery
  [merge-plugin] Merging npm-asset/punycode
  [merge-plugin] Running additional update to apply merge settings
    1/3:    http://packagist.org/p/provider-latest$7bd67f74c323ff6601aa1ca39c2b1fbd1bc7cc3e5cafedd8ff0e7988ccb7f647.json
    2/3:    http://packagist.org/p/provider-2017-07$00a09e1025fd22008b01aba5fba08327ccb5a3b7a4fcf1d27f25033dec2e0cf9.json
    3/3:    http://packagist.org/p/provider-2018-01$1f72d71f41de15a06d4105ec1a277fa32cc295e41b7ed9519fecf3704de7ddd6.json
    Finished: success: 3, skipped: 0, failure: 0, total: 3
  [merge-plugin] Loading vendor/yiisoft/yii2-jquery/composer.assets.json...
  [merge-plugin] Merging npm-asset/jquery
  [merge-plugin] Merging npm-asset/punycode
  [merge-plugin] Loading -dev sections of vendor/yiisoft/yii2-jquery/composer.assets.json...
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Dependency resolution completed in 0.000 seconds
Analyzed 77 packages to resolve dependencies
Analyzed 137 rules to resolve dependencies
Nothing to install or update
Generating autoload files
  [merge-plugin] Already merged vendor/yiisoft/yii2-jquery/composer.assets.json completely

This actually looks good when reading the messages:

For some reason the log says:

Installing dependencies (including require-dev) from lock file

Looks like an install, but that should be an update.

But here the additional packages are not installed, I need another run for that:

root@b620651b73b6:/app# composer update -vv
    1/3:    http://packagist.org/p/provider-latest$7bd67f74c323ff6601aa1ca39c2b1fbd1bc7cc3e5cafedd8ff0e7988ccb7f647.json
    2/3:    http://packagist.org/p/provider-2017-07$00a09e1025fd22008b01aba5fba08327ccb5a3b7a4fcf1d27f25033dec2e0cf9.json
    3/3:    http://packagist.org/p/provider-2018-01$1f72d71f41de15a06d4105ec1a277fa32cc295e41b7ed9519fecf3704de7ddd6.json
    Finished: success: 3, skipped: 0, failure: 0, total: 3
  [merge-plugin] Loading vendor/yiisoft/yii2-jquery/composer.assets.json...
  [merge-plugin] Merging npm-asset/jquery
  [merge-plugin] Merging npm-asset/punycode
  [merge-plugin] Loading -dev sections of vendor/yiisoft/yii2-jquery/composer.assets.json...
Loading composer repositories with package information
[...]
Updating dependencies (including require-dev)
Dependency resolution completed in 0.001 seconds
Analyzed 717 packages to resolve dependencies
Analyzed 504 rules to resolve dependencies
Package operations: 2 installs, 0 updates, 0 removals
Installs: npm-asset/jquery:3.2.1, npm-asset/punycode:1.3.2
  - Installing npm-asset/jquery (3.2.1): Loading from cache
 Extracting archive    REASON: Required by the root package: Install command rule (install npm-asset/jquery 3.2.1|install npm-asset/jquery 3.2.0|install npm-asset/jquery 3.1.1|install npm-asset/jquery 3.1.0|install npm-asset/jquery 2.2.4|install npm-asset/jquery 2.2.3|install npm-asset/jquery 2.2.2|install npm-asset/jquery 2.2.1|install npm-asset/jquery 2.2.0|install npm-asset/jquery 2.1.4|install npm-asset/jquery 2.1.3|install npm-asset/jquery 2.1.2|install npm-asset/jquery 2.1.1|install npm-asset/jquery 2.1.0|install npm-asset/jquery 1.12.4|install npm-asset/jquery 1.12.3|install npm-asset/jquery 1.12.2|install npm-asset/jquery 1.12.1|install npm-asset/jquery 1.12.0|install npm-asset/jquery 1.11.3|install npm-asset/jquery 1.11.2|install npm-asset/jquery 1.11.1|install npm-asset/jquery 1.11.0)

  - Installing npm-asset/punycode (1.3.2): Loading from cache
 Extracting archive    REASON: Required by the root package: Install command rule (install npm-asset/punycode 1.3.2|install npm-asset/punycode 1.3.1|install npm-asset/punycode 1.3.0)

Writing lock file
Generating autoload files
  [merge-plugin] Already merged vendor/yiisoft/yii2-jquery/composer.assets.json completely

Anything I have missed here?

CC: @cebe

schmunk42 commented 6 years ago

Addon: It works as expected, when I have no (or an empy) composer.lock file.

I just need to run update, additional files are found, merged and deps installed :+1: 

$ composer update -vv

[...]

Writing lock file
Generating autoload files
  [merge-plugin] Loading vendor/yiisoft/yii2-jquery/composer.assets.json...
  [merge-plugin] Merging npm-asset/jquery
  [merge-plugin] Merging npm-asset/punycode
  [merge-plugin] Running additional update to apply merge settings
  [merge-plugin] Loading vendor/yiisoft/yii2-jquery/composer.assets.json...
  [merge-plugin] Merging npm-asset/jquery
  [merge-plugin] Merging npm-asset/punycode
  [merge-plugin] Loading -dev sections of vendor/yiisoft/yii2-jquery/composer.assets.json...
Loading composer repositories with package information
Reading composer.json of yiisoft/yii2-jquery (feature/nam)
Importing branch feature/nam (dev-feature/nam)
Reading composer.json of yiisoft/yii2-jquery (feature/remove-pjax)
Importing branch feature/remove-pjax (dev-feature/remove-pjax)
Reading composer.json of yiisoft/yii2-jquery (master)
Importing branch master (dev-master)
Updating dependencies (including require-dev)
Dependency resolution completed in 0.001 seconds
Analyzed 718 packages to resolve dependencies
Analyzed 504 rules to resolve dependencies
Package operations: 2 installs, 0 updates, 0 removals
Installs: npm-asset/jquery:3.2.1, npm-asset/punycode:1.3.2
  - Installing npm-asset/jquery (3.2.1): Loading from cache
 Extracting archive  - Installing npm-asset/punycode (1.3.2): Loading from cache
 Extracting archiveWriting lock file
Generating autoload files
  [merge-plugin] Already merged vendor/yiisoft/yii2-jquery/composer.assets.json completely
schmunk42 commented 6 years ago

Halp! Can anyone shed some light on this?

bd808 commented 3 years ago

Having composer-merge-plugin install packages specified by files which are provisioned by other packages is not supported. This is a "chicken and egg" problem similar to the problem that composer-merge-plugin works around for it's own installation (see #170).

Composer-merge-plugin adds additional data to Composer's internal dependency resolver at the beginning of any Composer install/update action. Composer takes over from that point and actually installs the resolved packages. Any data that is not available on the local disk at the start of the install/update action is not available for either Composer or composer-merge-plugin to examine. If the use case of gathering configuration from packages as they are provisioned were to be supported, composer-merge-plugin would need to at minimum recompute its configuration following every package install and then trigger an additional Composer run if that configuration was different before & after the prior run. In the most pathological case this could trigger infinite Composer runs if each package provisioned a new configuration file that matched the composer-merge-plugin include files pattern and that config specified another package which did the same. Ultimately the problem is that we cannot traverse and verify the internal consistency of a configuration graph until that entire graph is known.