Versioning of internal modules API is not semantical

[LDVSOFT]

I'm reporting this issue in future-ideas so that some discussion could happen on the topic. It might be that it could be easily ruled out as out of scope, but I'd still be happy if any consideration would be done.

As well-known now, Jackson modules are a part of Jackson API that isn't as semantically versioned as the core modules:

• Modules must be minor-version aligned with core, some even assert that.

That essentially means that during project resolution adding a Jackson module for newer version of Jackson could break another module that target the older one by pulling newer core jar. That can be especially dangerous when loading all modules on the classpath, because in big projects there could be a lot of different Jackson users that rarely happen to agree on a version.

The issue is well-known in the wild, and many libraries because of that shade (via class relocation) their version of Jackson inside of them, increasing their package size and making security updates trigger cascades, without even touching on licensing questions. Also, some do a bad job at shading them properly and they still leak a bit.

One of the solutions is Maven dependency management with BOMs, as already proposed by Jackson, but that just overrides all versions, so it's good for enforcing a version but not for actually checking what version was needed in the first place. For example, if an update of a intermediate dependency would ask for a Jackson update you'd end up with LinkageErrors (happened to me!).

Another approach is implemented by Gradle build system where they propose to use platforms, and their example explicitly shows how to fix Jackson version resolution on client-side. Instead of just importing a given BOM version, it tells that actually every version of Jackson belongs to a platform, and Gradle could fetch BOMs for each of them to check versions or just check for version equality (via virtual platforms). Also, Gradle offers their own metadata solution so that published library have extra information that cannot be provided by native repository formats (i.e. ivy and maven), including statements that library is a part of the platform. Imagine pulling a Maven library would also pull a BOM in dependency management, for example.

So, if Jackson would be published with that information, Gradle users could fetch dependencies that would automatically align versions of Jackson artefacts to a properly working set. I don't know if such solution is even possible with good old Maven as I haven't used it in quite a while; I'm not even aware of tools to generate Gradle metadata without switching to Gradle in the first place. Also all non-Gradle users would not get anything too.

As a completely different approach, Jackson could provide more robust and stable facade for modules, so that using modules on other versions of core Jackson is possible. Downsides are, of course, costs of maintaining a stable internal interface. Given that package names are to change, though, maybe you could figure something out.

So, looks like there isn't an obvious solution for that. I hope someone can provide better ideas.

[pjfanning]

The jackson bom already exists - see https://mvnrepository.com/artifact/com.fasterxml.jackson/jackson-bom

Gradle modules are supported for most jackson artifacts - see https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.13.3/ (see the .module file)

There is no silver, in my opinion. In the end of the day, you need to have unit tests in your build that exercise your use of Jackson - so that if you upgrade jar dependencies, the new jars that are brought in as a result are tested together before you deploy to production.

[cowtowncoder]

Hi @LDVSOFT as @pjfanning pointed out, jackson-bom exists (so it's a good idea but already done). Gradle's GMM may or may not work, but it is also included in the later version.

Now: as you may or may not work, module version dependencies are asymmetric in that jackson-databind has pretty strong guarantees that a new jackson-databind should work with all newer module versions. But the reverse is not guaranteed; newer modules may well require recent jackson-databind. This because jackson-databind can (and does) add new functionality. This is not something that can really be changed unless preventing any additions (and most improvements) in minor versions.

So, for example, with hypothetical module jackson-module-demo and jackson-databind:

• jackson-module-demo version 2.5 should work with ALL jackson-databind versions 2.5 and above (but not with 2.4 or earlier). So increasing only jackson-databind version is typically safe because module depends on it (and not vice versa)
• jackson-module-demo 2.6 would not (be guaranteed to) work with jackson-databind 2.5 or earlier because it requires on 2.6 level module API. So upgrading that module does require upgrading databind along, in general -- some cases might still work but it is less likely

Still, the main problem with compatibility is not really Module API which is pretty stable. Typical compatibility problems (based on my experience) seem to be often related to:

1. Sub-classing (inheritance) of handlers types like standard JsonDeserializer / JsonSerializer and overriding of methods that are not part of API
2. Fixes to behavior where existing usage was reliant on faulty (but existing) behavior
3. Unintended behavior changes to functionality not covered by unit tests. Essentially ANY behavior not covered by unit tests has to be considered "undefined" -- it would be great if there were more unit test contributions to cover less common use cases. This would help prevent unintended behavior changes