The number of dependencies and speed performance issue

[redthor]

Hi there,

We're moving to a monorepo/manyrepo structure using splitsh-lite. We have tried using the merge plugin to grab all the composer files of our many repos and bring them into the root composer file but for some reason this makes things really slow.

Conceptually I would have thought that bringing in all the dependencies from the separate composer files would not change the work that composer would need to do?

Set up

Original structure A:

• main app with many composer dependencies (our in-house packages)
• multiple package repositories, each with their own composer.json

New structure B:

• main app with all our package dependencies removed
• package dependencies brought into ./packages/* folder into main app repo
• composer merge plugin installed
• composer merge config:

    "extra": {
        "merge-plugin": {
            "require": ["./packages/*/composer.json"],
            "ignore-duplicates": true,
            "merge-dev": false
        }
    },

Results

time composer update -v --acpu-autoloader

Stats	A - Orig	B - With Merge Plugin
1st Dependency Resolution	2.350 seconds	3.425 seconds
Analyzed packages	23115	40488
Analyzed rules	1231598	2059645
2nd Dependency Resolution	0.002 seconds	0.001 seconds
Total time	4m21.671s	25m0.974s

Does anyone have any idea what might be happening? thanks.

[redthor]

I've created a log file for each with composer update -vvv --apcu-autoloader > composer.log 2>&1 and I find that our packages are being downloaded over and over again like this:

Downloading https://<repo>/biz/package-1.json
Downloading https://<repo>/biz/package-1.json
Downloading https://<repo>/biz/package-1.json
Downloading https://<repo>/biz/package-1.json

The same package downloaded over 20 times, for every package.

[redthor]

I've installed https://github.com/hirak/prestissimo and it seems to help. Though I suspect it might just be doing the duplicate downloads in parallel.