Autoloader: A potential issue with different major versions of dependencies

[anomiex]

I looked into the Jetpack autoloader when doing some reviews. It seems like an interesting approach to the problem of trying to drop plugins, each with their own vendor directory full of dependencies, into an existing installation without requiring a top level composer install to properly collect all the dependencies into one vendor directory.

For example, if a plugin uses "some/library" at version 1.0.3, and another plugin uses "some/library" at version 1.0.7, our autoloader will pick the 1.0.7 version.

But consider if a third plugin brings in "some/library" at version 2.0.0, which (as indicated by the semver version number) has breaking changes with respect to the 1.x series. Our autoloader would pick 2.0.0, which might break one of the first two plugins if they use something from the library that was removed or changed.

Remember that even if we avoid this in our own libraries by releasing a "some/libraryV2" instead of a 2.0.0 version, the autoloader may also be used to bring in other libraries that likely don't do that sort of thing.

Composer itself handles this sort of situation by raising an error when you try to composer install or composer update, complaining that no version satisfies all the dependencies. The third plugin might even avoid the problem by depending on "~1.0.7 || ~2.0.0" (and doing library feature detection somehow in-code).

At the least, we might log a warning of some sort if we have to choose between different major versions. Ideally we might collect the constraints applying to each package in addition to the package's version in the generated jetpack_autoload_*.php, and use that when autoloading to select the best version of the package (or raise an error if no version satisfies all constraints) instead of just picking the highest.

[jeherve]

cc @kbrown9 for your thoughts on this.

[kbrown9]

For example, if a plugin uses "some/library" at version 1.0.3, and another plugin uses "some/library" at version 1.0.7, our autoloader will pick the 1.0.7 version.

But consider if a third plugin brings in "some/library" at version 2.0.0, which (as indicated by the semver version number) has breaking changes with respect to the 1.x series. Our autoloader would pick 2.0.0, which might break one of the first two plugins if they use something from the library that was removed or changed.

Yes, this is a major limitation of Jetpack's autoloader.

In addition, Jetpack's autoloader could lose control of which package version is loaded when another plugin uses a package but does not use Jetpack's autoloader. For example, if a fourth plugin installs "some/library" version 1.0.2, but does not use Jetpack's autoloader, version 1.0.2 files from "some/library" could be loaded if the fourth plugin is loaded before the others.

At the least, we might log a warning of some sort if we have to choose between different major versions. Ideally we might collect the constraints applying to each package in addition to the package's version in the generated jetpack_autoload_*.php, and use that when autoloading to select the best version of the package (or raise an error if no version satisfies all constraints) instead of just picking the highest.

Collecting version constraints, attempting to resolve them, and displaying a message when version constraints cannot be resolved are long-term goals for the Jetpack autoloader. As far as I know, no one is working on it yet. If you're interested in that, feel free to work on it!

[anomiex]

If you're interested in that, feel free to work on it!

I might just do that at some point. Some notes/brainstorming:

• Composer doesn't seem to directly record the constraints that led to a package being selected. Looks like we may be able to extract them by starting from the "root" package object and collecting ->getRequires() recursively (->getTarget() and ->getConstraint()). And maybe also ->getDevRequires() on the root package only? Note a package may be required via multiple paths with different constraints.
• It may be more efficient to restructure the manifests by package, with each package having the version, constraints, and maps, versus the current strategy of storing the maps with a version added to each individual map entry. That would necessitate an autoloader V3 though.
• The constraint checking should be able to be more straightforward than what Composer itself has to do, because we can probably get away with caring only about the individual package rather than the whole dependency tree.
• Too bad we can't just re-run Composer on plugin install/update/removal to create one centralized vendor directory. But I agree with this comment where it points out that a site may have permissions that let someone drop in plugins but will not allow WordPress to modify vendor/. That would probably also break WordPress's and Jetpack's updating, but the site admin who set it up that way might consider that a feature.