[JENKINS-49651] Extend plugin/update center metadata with known incompatibilities

[timja]

The current plugin metadata allows it to express minimum required versions of core and various other plugins, and this information is mirrored in update center metadata. That is enough for typical scenarios involving compatible upgrades of API components.

However, we have repeatedly encountered cases where Jenkins developers know that there are combinations of software components which will be broken for users. These include:

• Removal of deprecated and unsupportable APIs (such as Java methods). We can update all "known" plugins which were using those APIs, but we need to force users to install those updates if they also update the API (core or plugin) component.
• Major refactorings such as SCM API 2.x which must be upgraded atomically.
• Behavioral incompatibilities forcing plugin updates, such as cases found in JEP-200.
• "Tombstone" releases of plugins in which a plugin is replaced by another—we typically wish to force users to update to the tombstone if they are updating related components.

What all of these cases have in common is that there is some manner of incompatibility, and thus users who upgrade some components must also upgrade other components which would normally be in an inverse dependency relationship (or no expressed dependency at all) to avoid problems.

Unlike regular plugin dependencies, these relationships cannot be expressed in the downstream plugin manifest. For example, the fact that api 2.0 made incompatible changes which would break installations of impl 1.3 (which depended on api 1.2) cannot be expressed in impl 1.3's manifest—this might have been released years before the rewrite of api was even conceived. It is only the release of impl 2.0 which reveals the upgrade path. So we have two options:

• Keep the incompatibility marking in the upstream plugin manifest (or some similar metadata resource in the case of core). For example, api 2.0's manifest could express something like:

    Incompatible-With: impl < 2.0, other-impl < 2.0, ...
    

  This has the advantage that the relationships can be enforced by any part of the system, including plugin loading during startup, Docker image builders, JEP-201 if extended to declarative plugin configuration, etc. On the other hand you would need to a release an update to api every time another incompatible implementation is discovered (and a release using api 2.x cut).

• Advertise all this metadata in update-center.json, like we do for example with notifications of plugin releases containing disclosed security vulnerabilities. The advantage is that we can easily publish new information as it arrives. On the other hand, the metadata is only easily available to the update center code and thus applicable only to traditional GUI upgrades.

Users participating in JEP-301 "Evergreen" should have less need of a system like this, since they would automatically receive only combinations of components which were pretested and verified compatible. But even Evergreen users could benefit if they are also adding plugins from outside the Essentials set, which might well be known to be broken by changes being made to other components (inside or outside Essentials).

---

Originally reported by jglick, imported from: Extend plugin/update center metadata with known incompatibilities

• status: Open
• priority: Major
• resolution: Unresolved
• imported: 2022/01/10

[timja]

jglick:

Another use case: for JEP-200 we added several whitelist entries to core that actually apply to classes bundled in plugins, to make sure people could do a core upgrade without having to know about plugins to update. With this API, once the plugin releases appear which (for now redundantly) include those entries, we could delete them from core with a notice that the newer core versions are incompatible with the older plugin versions.

[timja]

cobexer:

Maybe go shopping in the RPM dependency documentation for this: http://rpm.org/user_doc/dependencies.html http://rpm.org/user_doc/more_dependencies.html http://rpm.org/user_doc/boolean_dependencies.html

And here is a guide from openSUSE on how to do package splits, merges, renames, ...:

https://en.opensuse.org/openSUSE:Package_dependencies#Basic_dependencies

 

[timja]

jglick:

Yes Linux package manager docs would be a good source of potential use cases to validate a solution against. I think we can get away with a simpler system, since:

• Jenkins does not really need to deal with platform-specific issues.
• Most plugin development happens under a single organization’s umbrella. Even if plugin maintainers operate autonomously they are still delivering software targeted toward a core produced by that same organization and using tools and an update center maintained by the same core developers. This is a much simpler situation than Linux, in which both the kernel and various userspace software projects are maintained by completely disparate groups, and multiple distributions compete to package those components independently, picking versions and backports, with these decisions often being made by people other than the upstream developer.
• Evergreen reduces the pressure to support exotic scenarios that might occur with people running the many LTS flavors offered by various Linux distributions. The Jenkins project produces only one official “long-term” support line, and (until Evergreen) no “pinned” plugin versioning at all.

[timja]

cobexer:

I think we can get away with a simpler system

Sure, I just think that with Requirements and Incompatible-With (aka Conflicts) you almost have everything in place to enable:

• Renames
• Splits
• Merges

So I added the RPM documentation links here to save the implementer the time to come up with compatible semantics themselves.

Thinking about this some more I think Obsoletes and also handling core updates in the same resolver would net you almost everything. (IIRC KK talked about wanting to update core like the plugins)

[timja]

jglick:

JENKINS-31096 is another example of where this would be useful.

[timja]

jglick:

JENKINS-41854 is another example (DynamicContext needed a workflow-cps update to be interpreted).

[timja]

jglick:

JENKINS-5303 would be a major use case.