[composer] root package version not detected, preventing any updates when packages conflict

danepowell commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Package ecosystem

composer

Package manager version

Composer 2

Language version

PHP 7.4 (enforced by config: platform in composer.json)

Manifest location and content before the Dependabot update

https://github.com/acquia/blt/blob/main/composer.json

dependabot.yml content

https://github.com/acquia/blt/blob/main/.github/dependabot.yml

Updated dependency

"symfony/console": "^4.4.6 || ^6",

Current version is 4.4.48, 4.4.49 and 6.2.1 are available.

What you expected to see, versus what you actually saw

I expected dependabot to update symfony/console to 4.4.49, the latest available version compatible with my platform constraints. Instead, it proposed and rejected updating to 6.2.1 and ignored the fact that 4.4.49 exists at all.

Native package manager behavior

composer update correctly updates symfony/console to 4.4.49.

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

danepowell commented 1 year ago

The problem may be more extensive than I first believed. We are on Drush 11.3.2, but 11.4.0 was just released and Dependabot refuses to update to it. 11.4.0 is the latest available version and fully compatible with our codebase.

updater | INFO <job_549713423> Requirements to unlock update_not_possible
updater | INFO <job_549713423> Requirements update strategy bump_versions_if_necessary
updater | INFO <job_549713423> No update possible for drush/drush 11.3.2

It appears that Dependabot was working as recently as Oct 31 for us. Sometime between then and now something broke it.

jeffwidman commented 1 year ago

👋 Big fan of Acquia... I used your platform for a project several years ago and had a very positive experience.

I expected dependabot to update symfony/console to 4.4.49, the latest available version compatible with my platform constraints. Instead, it proposed and rejected updating to 6.2.1 and ignored the fact that 4.4.49 exists at all.

Can you paste the relevant job logs for this? I suspect it's a different reason than the Drush update. This may be related to https://github.com/dependabot/dependabot-core/issues/6289.

For Drush, that update_not_possible is a catch-all for a whole mess of errors... including some that composer spits out. Does Drush work fine via composer update?

If you haven't seen it, https://github.com/dependabot/dependabot-core/#running-with-docker can be very helpful to quickly replicate within docker what Dependabot is doing and then add a few puts statements + comment out code to figure out the underlying issue.

danepowell commented 1 year ago

Awesome, glad you had a great experience with Acquia!

I'm also so glad the dependabot core is open source and can be run via Docker. Indeed this looks like a problem exacerbated by #6289. Running DEBUG_HELPERS=true bin/dry-run.rb composer acquia/blt revealed the previously-hidden error:

=== drush/drush (11.3.2)
 => checking for updates 8/22
🌍 https://repo.packagist.org/p/drush/drush.json
 => latest available version is 11.4.0
php -d memory_limit=-1 /opt/composer/v2/bin/run
{"error":"Your requirements could not be resolved to an installable set of packages.\n  Problem 1\n    - acquia\/blt is present at version 1.0.0+no-version-set and cannot be modified by Composer\n    - acquia\/drupal-environment-detector 1.5.2 conflicts with acquia\/blt <13.5.2.\n    - acquia\/drupal-environment-detector is locked to version 1.5.2 and an update of this package was not requested.\n"}
PHP Deprecated:  Installer::setIgnorePlatformRequirements is deprecated since Composer 2.2, use setPlatformRequirementFilter instead. in /opt/composer/v2/vendor/composer/composer/src/Composer/Installer.php on line 1289
 => latest allowed version is 11.3.2
php -d memory_limit=-1 /opt/composer/v2/bin/run
{"error":"Your requirements could not be resolved to an installable set of packages.\n  Problem 1\n    - acquia\/blt is present at version 1.0.0+no-version-set and cannot be modified by Composer\n    - acquia\/drupal-environment-detector 1.5.2 conflicts with acquia\/blt <13.5.2.\n    - acquia\/drupal-environment-detector is locked to version 1.5.2 and an update of this package was not requested.\n"}
PHP Deprecated:  Installer::setIgnorePlatformRequirements is deprecated since Composer 2.2, use setPlatformRequirementFilter instead. in /opt/composer/v2/vendor/composer/composer/src/Composer/Installer.php on line 1289
 => requirements to unlock: update_not_possible
 => requirements update strategy: bump_versions_if_necessary
    (no update possible 🙅‍♀️)

It looks like the problem is that Composer isn't correctly detecting the version of the root package (acquia/blt) and assumes it to be 1.0.0, and a dependency (acquia/drupal-environment-detector) conflicts with BLT < 13.5.2. Composer thus sees a conflict with any potential update and throws the error.

I don't fully understand how Composer detects package versions, but I think the solution would be for Dependabot to clone enough Git history for Composer to be able to infer the version correctly.

Links to BLT and Drupal Environment Detector composer.json dependencies causing the conflict:

danepowell commented 1 year ago

I fixed this by changing the acquia/drupal-environment-detector conflict config from "acquia/blt": "<13.5.2" to "acquia/blt": ">=12.0.0 <13.5.2". That way, when dependabot updates BLT dependencies, the inferred BLT version (1.0.0) no longer conflicts.

But this remains an issue for Dependabot and possibly Composer to address. My guess is that Dependabot uses a shallow clone with no branch/tag information that causes Composer to infer the wrong package version. But possibly Composer should also ignore the situation where a dependent package conflicts with the root package due to an inferred root package version.

weirdan commented 1 year ago

This is actually a manifestation of #3888. Composer provides an escape hatch in the form of COMPOSER_ROOT_VERSION env variable, but dependabot does not let us set it.

dependabot / dependabot-core