A gradle plugin to ensure your dependency versions are consistent across all subprojects, without requiring you to hunt down and force every single conflicting transitive dependency.
Direct dependencies are specified in a top level versions.props
file and then the plugin relies on Gradle constraints to figure out sensible versions for all transitive dependencies - finally the whole transitive graph is captured in a compact versions.lock
file.
Apply the plugin (root project only):
plugins {
id "com.palantir.consistent-versions" version "<current version>"
}
You can find the current version under releases.
In one of your build.gradle files, define a versionless dependency on some jar:
apply plugin: 'java'
dependencies {
implementation 'com.squareup.okhttp3:okhttp'
}
Create a versions.props
file and provide a version number for the jar you just added:
com.squareup.okhttp3:okhttp = 3.12.0
Run ./gradlew writeVersionsLocks
and see your versions.lock file be automatically created. This file should be checked into your repo:
# Run ./gradlew writeVersionsLocks to regenerate this file
com.squareup.okhttp3:okhttp:3.12.0 (1 constraints: 38053b3b)
com.squareup.okio:okio:1.15.0 (1 constraints: 810cbb09)
nebula.dependency-recommender
dependencyRecommendations.getRecommendedVersion
-> getVersion
Many languages have arrived at a similar workflow for dependency management: direct dependencies are specified in some top level file and then the build tool figures out a sensible version for all the transitive dependencies. The whole dependency graph is then written out to some lock file. We asked ourselves, why doesn't this exist for Java?
package.json
and then yarn auto-generates a yarn.lock
file.cargo.toml
file and have cargo.lock
auto-generated.Gopkg.toml
file and have Gopkg.lock
auto-generated.Gemfile
file and have Gemfile.lock
auto-generated.Specifically this plugin delivers:
failOnVersionConflict()
does the job, but it comes with some significant downsides - see below). With gradle-consistent-versions, dependencies across all your subprojects have a sensible version provided.versions.lock
.nebula.dependency-recommender
nebula.dependency-recommender pioneered the idea of 'versionless dependencies', where gradle files just declare dependencies using compile "group:name"
and then versions are declared separately (e.g. in a versions.props
file). Using failOnVersionConflict()
and nebula.dependency-recommender's OverrideTransitives
ensures there's only one version of each jar on the classpath.
nebula.dependency-recommender forces all your dependencies, which overrides all version information that libraries themselves provide.
Unfortunately, failOnVersionConflict means developers often pick conflict resolution versions out of thin air, without knowledge of the actual requested ranges. This is dangerous because users may unwittingly pick versions that actually violate dependency constraints and may break at runtime, resulting in runtime errors suchs as ClassNotFoundException
, NoSuchMethodException
etc
Specify versions for your direct dependencies in a single root-level versions.props
file. Think of these versions as the minimum versions your project requires.
com.fasterxml.jackson.*:jackson-* = 2.9.6
com.google.guava:guava = 21.0
com.squareup.okhttp3:okhttp = 3.12.0
junit:junit = 4.12
org.assertj:* = 3.10.0
The * notation ensures that every matching jar will have the same version - they will be aligned1 using a virtual platform.
Note that this does not force okhttp to exactly 3.12.0, it just declares that your project requires at least 3.12.0. If something else in your transitive graph needs a newer version, Gradle will happily select this. See below for how to downgrade something if you really know what you're doing.
[1]: If multiple lines from versions.props
match a particular jar, the most specific one will be chosen (the one with the most characters being different from *
).
This has the side effect that a line referring specifically to a jar is independent, and that jar's version never gets aligned to the versions of other jars, even if there are other lines containing *
which would otherwise match that jar.
When you run ./gradlew --write-locks
, the plugin will automatically write a new file: versions.lock
which contains a version for every single one of your transitive dependencies.
Notably, this lockfile is a compact representation of your dependency graph as it just has one line per dependency (unlike nebula lock files which spanned thousands of lines).
javax.annotation:javax.annotation-api:1.4.0 (3 constraints: 1a310f90)
javax.inject:javax.inject:1 (2 constraints: d614a0ab)
javax.servlet:javax.servlet-api:3.1.0 (1 constraints: 830dcc28)
javax.validation:validation-api:1.1.0.Final (3 constraints: dc393f20)
javax.ws.rs:javax.ws.rs-api:2.0.1 (8 constraints: 7e9ce067)
[Test dependencies]
cglib:cglib-nodep:3.1 (1 constraints: 2a0e1330)
com.github.zafarkhaja:java-semver:0.9.0 (1 constraints: c315c0d2)
com.jayway.awaitility:awaitility:1.6.5 (1 constraints: c615c1d2)
The lockfile sources production dependencies from the compileClasspath and runtimeClasspath configurations, and
test dependencies from the compile/runtime classpaths of any source set named jmh
or whose name ends in "test"
(e.g. test
, integrationTest
, eteTest
).
There is a verifyLocks
task (automatically run as part of check
) that will ensure versions.lock
is still consistent
with the current dependencies.
To understand why a particular version in your lockfile has been chosen, run ./gradlew why --dependency <dependency>
to expand the constraints:
> Task :why
com.fasterxml.jackson.core:jackson-databind:2.9.8
com.fasterxml.jackson.module:jackson-module-jaxb-annotations -> 2.9.8
com.netflix.feign:feign-jackson -> 2.6.4
com.palantir.config.crypto:encrypted-config-value -> 2.6.1
com.palantir.config.crypto:encrypted-config-value-module -> 2.6.1
This is effectively just a more concise version of dependencyInsight
:
./gradlew dependencyInsight --configuration unifiedClasspath --dependency jackson-databind
checkUnusedConstraints
prevents unnecessary constraints from accruing in your versions.props
file. Run
./gradlew checkUnusedConstraints --fix
to automatically remove any unused constraints from your props file.
If you want to use the resolved version of some dependency elsewhere in your Gradle files, gradle-consistent-versions offers the getVersion(group, name, [configuration])
convenience function. For example:
task demo {
doLast {
println "We chose guava " + getVersion('com.google.guava', 'guava')
}
}
This function may not be invoked at Gradle Configuration time as it involves resolving dependencies. Put it inside a closure or provider to ensure it is only invoked at Execution time.
By default, this function resolve the unifiedClasspath
configuration to supply a version, but you can always supply a different configuration if you want to:
task printSparkVersion {
doLast {
println "Using spark version: " + getVersion('org.apache.spark', 'spark-sql_2.11', configurations.spark)
}
}
Gradle has first-class support for sourcing version constraints from published BOMs so they work fine with gradle-consistent-versions:
allprojects {
apply plugin: 'java-base'
dependencies {
rootConfiguration platform('com.foo.bar:your-bom')
}
}
Make sure you apply BOMs within an allprojects
closure, as gradle-consistent-versions must be able to unify constraints from all subprojects.
Note: java-base
is necessary, even on projects that don't have java source code, otherwise gradle will silently interpret the platform(...)
dependency as if it was a normal library dependency, and will not import the constraints from that BOM.
The preferred way to control your dependency graph is using dependency constraints on gradle-consistent-versions' rootConfiguration
. For example:
dependencies {
constraints {
rootConfiguration 'org.conscrypt:conscrypt-openjdk-uber', {
version { strictly '1.4.1' }
because '1.4.2 requires newer glibc than available on Centos6'
}
rootConfiguration 'io.dropwizard.metrics:metrics-core', {
version { strictly '[3, 4[' }
because "Spark still uses 3.X, which can't co-exist with 4.X"
}
}
}
Gradle will fail if something in your dependency graph is unable to satisfy these strictly constraints. This is desirable because nothing is forced in your transitive graph.
If you discover a bug in some library on your classpath, the recommended approach is to use dependencyInsight to figure out why that version is on your classpath in the first place and then downgrade things until that library is no longer brought in. Once the dependency is gone, you can specify a rootConfiguration constraint to make sure it doesn't come back (see above).
./gradlew dependencyInsight --configuration unifiedClasspath --dependency retrofit
Occasionally however, downgrading things like this is not feasible and you just want to force a particular transitive dependency. This is dangerous because something in your transitive graph clearly compiled against this library and might be relying on methods only present in the newer version, so forcing down may result in NoSuchMethodErrors at runtime on certain codepaths.
allprojects {
configurations.all {
resolutionStrategy {
force 'com.squareup.retrofit2:retrofit:2.4.0'
}
}
}
Developers usually want just one SLF4J implementation on the classpath. If some of your dependencies rely on their own logging implementations (e.g. commons-logging or log4j), you can use the following snippet to ensure that all logging will go through SLF4J.
allprojects {
dependencies {
modules {
module('commons-logging:commons-logging') {
replacedBy('org.slf4j:jcl-over-slf4j', 'slf4j allows us supply our own implementation')
}
module('log4j:log4j') {
replacedBy('org.slf4j:log4j-over-slf4j', 'slf4j allows us supply our own implementation')
}
}
}
}
We've seen the following error when using dependencySubstitution:
> Could not find method module() for arguments [org.glassfish.hk2.external:javax.inject] on configuration ':my-project:subprojectUnifiedClasspathCopy' of type org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.
Adding explicit it
calls works around this error:
configurations.configureEach {
resolutionStrategy.dependencySubstitution {
- substitute module('org.glassfish.hk2.external:javax.inject') with module('javax.inject:javax.inject:1')
+ it.substitute it.module('org.glassfish.hk2.external:javax.inject') with it.module('javax.inject:javax.inject:1')
}
}
Sometimes, devs have multiple test projects (unit tests, integration tests) that use a subset of common test classes.
* :foo
\--- source set 'test'
\--- :foo-test-common
* :foo-integration
\--- source set 'integrationTest'
\--- :foo-test-common
* :foo-test-common
\--- source set 'main'
In this case, we'd like to prevent GCV from locking :foo-test-common
's main
source set to production dependencies,
and instead treat the entire project as test dependencies. We can do this via the following snippet:
# foo-test-common/build.gradle
apply plugin: 'java'
versionsLock {
testProject()
}
In order for this plugin to function, we must be able to guarantee that no dependencies are resolved at configuration time. Gradle already recommends this but gradle-consistent-versions enforces it.
In many cases, it's just a matter of using a closure or a provider, for example:
task copySomething(type: Copy) {
- from configurations.spark.singleFile // 🌶🌶🌶 this downloads spark at configuration time, slowing down every `./gradlew` invocation!
+ from { configurations.spark.singleFile }
into "$buildDir/foo/bar"
}
Due to an implementation detail of this plugin, we require settings.gradle to declare a rootProject.name
which is unique.
+rootProject.name = 'tracing-root'
-rootProject.name = 'tracing'
include 'tracing'
include 'tracing-api'
include 'tracing-jaxrs'
include 'tracing-okhttp3'
include 'tracing-jersey'
include 'tracing-servlet'
include 'tracing-undertow'
By default, this plugin will apply the constraints from versions.props
to all configurations.
To exclude a configuration from receiving the constraints, you can add it to excludeConfigurations
, configurable through the versionRecommendations
extension (in the root project):
versionRecommendations {
excludeConfigurations 'zinc'
}
Using a combination of automation and some elbow grease, we've migrated ~150 projects from nebula.dependency-recommender
to com.palantir.consistent-version
:
plugins {
+ id 'com.palantir.consistent-versions' version '<current version>'
}
allprojects {
dependencies {
+ rootConfiguration platform('com.palantir.witchcraft:witchcraft-core-bom')
}
- dependencyRecommendations {
- mavenBom module: 'com.palantir.witchcraft:witchcraft-core-bom'
- }
- configurations.all {
- resolutionStrategy {
- failOnVersionConflict()
- }
- }
}
You can also likely delete the 'conflict resolution' section of your versions.props.
GCV will just work out of the box with Baseline 3.0.0 and newer.
If still using Baseline 2.x, then you'll need to disable its own versions plugin which conflicts with GCV:
Add the following to your gradle.properties
fully turn off nebula.dependency-recommender (only necessary if you use com.palantir.baseline
):
org.gradle.parallel=true
+com.palantir.baseline-versions.disable=true
dependencyRecommendations.getRecommendedVersion
-> getVersion
If you rely on this Nebula function, then gradle-consistent-versions has a similar alternative:
-println dependencyRecommendations.getRecommendedVersion('com.google.guava:guava')
+println getVersion('com.google.guava:guava')
Note that you can't invoke this function at configuration time (e.g. in the body of a task declaration), because the plugin needs to resolve dependencies to return the answer and Gradle strongly discourages resolving dependencies at configuration time.
Alternatives:
productDependencies
, use detectConstraints = true
or upgrade to 3.Xfrom
or to
parameters of a Copy
task, use a closure or move the whole thing into a doLast block. task copySomething(type: Copy) {
- from "$buildDir/foo/bar-${dependencyRecommendations.getRecommendedVersion('group', 'bar')}"
+ from { "$buildDir/foo/bar-${getVersion('group:bar')}" }
...
No. We tried Gradle 4.8's first-class lockfiles, but found a critical usability problem: it allowed semantic merge conflicts. For example:
gradle/dependency-locks/x.lockfile
Concrete example:
The badness arises when:
Our format avoids this problem by adding some redundant information about the dependency graph so that changes like these will result in pre-merge git conflicts instead of post-merge semantic conflicts.
Our format extends gradle's lockfiles format by writing down who wanted each component (com.google.guava:guava
) and which version they requestedm, e.g.
com.google.guava:guava:18.0 (2 constraints 4a440103)
The hash, 4a440103
, is derived from Guava's dependents:
com.fasterxml.jackson.datatype:jackson-datatype-guava -> 18.0
com.github.rholder:guava-retrying -> [10.+,)
This ensures that if two PRs affect the set of constraints on guava they will result in a git conflict, which prevents the semantic merge conflicts described above and ensures develop won't break:
com.google.guava:guava:10.0 (1 constraints: 50295fc1)
The hash from one side of the merge confict (f59715c4
) came from adding a tracing dependent:
com.fasterxml.jackson.datatype:jackson-datatype-guava -> 18.0
+com.palantir.tracing:tracing -> 16.0
com.github.rholder:guava-retrying -> [10.+,)
While the other hash 50295fc1
, is derived by deleting jackson, leaving just a single dependent:
-com.fasterxml.jackson.datatype:jackson-datatype-guava -> 18.0
com.github.rholder:guava-retrying -> [10.+,)
With vanilla Gradle 4.8 lockfiles, this scenario would have merged but then failed on develop due to an inconsistent lockfile.
Internally, a few projects started using Nebula's Gradle Dependency Lock Plugin but eventually abandoned it because it introduced too much dev friction. Key problems:
Both of these problems are solved by this plugin because the new gradle lock files are extremely compact (one line per dependency) and they can never get out of date because gradle validates their correctness every time you resolve the unifiedClasspath configuration.