Closed ianbrandt closed 1 year ago
Unfortunately gradle module metadata and its usages are very flaky. This is somewhat on Gradle having to evolve with minimal compatibility breaks, but most often due to other plugins intentionally misbehaving. You are welcome to debug our Resolver to see if you can diagnose the root problem.
In similar cases that I have debugged, it has usually been the Kotlin or Android plugins which violate the Gradle APIs. They do hack to the internals to bypass restrictions, hard code configuration names instead of using metadata (e.g. breaking configuration copying or extends), reinvent basic features in brittle ways (e.g. default dependencies), and simply don't want to play nice or respect others in the community. Where possible we have some workarounds to fix their bugs, but often I don't know how to fix an issue when the other plugins refuse to correct their broken behavior. This plugin uses Gradle's apis in very simple ways that held true since Gradle 1.0, approaches approved by the core team, and other than slight compatibility changes between major versions it is rarely at fault.
For BOMs the Spring team was great when their plugin broke compatibility and fixed that mistake (https://github.com/spring-gradle-plugins/dependency-management-plugin/issues/77). My expertise in Gradle is aging and the interactions are becoming complex, so unfortunately I'm probably not the best one to help you figure this out anymore. But you are welcome to dig in by using a local build of the plugin to try to see where the problem comes from and if something on our side, then PRs are always welcome.
I could start by converting the reproducer project to be Java only, and see if removing the Kotlin Gradle Plugin makes any difference.
Somewhat encouraging, I follow and vote for all the Gradle compatibility issues for Kotlin (e.g. https://youtrack.jetbrains.com/issue/KT-54231/Compatibility-with-Gradle-8.0-release, which links to similar meta issues for the earlier and later versions), and they seem to be making a significant push lately to catch up and correct things.
Done:
https://github.com/ianbrandt/gradle-versions-plugin-platform-constraints/tree/java-only
I get basically the same result:
The following dependencies have later milestone versions:
- org.springframework.boot:org.springframework.boot.gradle.plugin [2.7.8 -> 3.0.2]
https://spring.io/projects/spring-boot
Failed to determine the latest version for the following dependencies (use --info for details):
- org.apache.logging.log4j:log4j-core
2.19.0
- org.springframework:spring-context
6.0.4
- org.springframework.boot:spring-boot
3.0.2
- org.springframework.boot:spring-boot-autoconfigure
3.0.2
- org.springframework.boot:spring-boot-starter
3.0.2
- org.springframework.boot:spring-boot-starter-log4j2
3.0.2
- org.springframework.boot:spring-boot-starter-test
- org.springframework.boot:spring-boot-starter-webflux
3.0.2
thanks! I'll try to spend some time this weekend and see if anything jumps out at me.
Might be a regression here. Seeing the same on 0.45.0, no problems on 0.44.0. We are running against a nexus on AWS, if that helps.
Thanks for that insight @mikand13! I can confirm that the sample project resolves fine in 0.44, with the small caveat of log4j which might be a configuration mistake. I'll bisect to see what might have caused this.
Hi @jvandort,
It seems that your change in #721 to support Gradle 9.0's changes is the cause of this failed resolution. Can you please take a look and advise? I was able to confirm by taking your commit and the one prior, publishing versions to the local maven repository, and specifying those in the sample java project above. With your changes we observe the reported errors. Thanks!
Thanks for letting me know. I'll take a look sometime this week and will let you know what I find
It seems the problem lies here:
Incompatible because this component declares an API of a component compatible with Java 17 and the consumer needed a runtime of a component compatible with Java 11
Its understandable that this error is hard to decode. We've heard a lot of complaints about the No matching variant of X was found
error, for good reason. The error message is very wordy and we have some tentative plans to update the message, though we do not have a concrete timeline to update it.
However, the root of the problem seems to be that the project is java 11, but the newest version of spring is java 17 compatible only. Essentially, gradle is failing to resolve the newest version of spring since a java 11 project is not allowed to depend on java 17 dependencies.
I guess the real question is, what behavior should this plugin have when such a situation occurs? Should the plugin recommend the newest version which is compatible with the current java version? Thinking about that right now, that would seemingly be quite difficult with how things work now. Or, should the plugin recommend the newest version outright even if it is not compatible with the newest version? What is the current behavior?
I have not tested this yet, but you should be able to call JavaPluginExtension#disableAutoTargetJvm
to disable the behavior where Gradle attempts to do this Java version detection. However, this could be risky, as it affects the entire gradle project.
Alternatively, you could try to unset the TargetJvmVersion.TARGET_JVM_VERSION_ATTRIBUTE
attribute on the configuration being resolved so that we don't do the checking. For context, this is the attribute which the auto-target-jvm functionality is setting itself. Though, looking at the code, simply unsetting this may not work, as the attribute is only set when the configuration is locked from mutation -- which happens during resolution, after the plugin has executed. One option would be to set the attribute value to `0, since the auto-target-jvm functionality does not override a value for the attribute if it has been set already.
Context:
https://github.com/gradle/gradle/blob/da59fbd0db73df45e2c59b2aca7cda83d62bd89f/subprojects/core/src/main/java/org/gradle/api/internal/artifacts/JavaEcosystemSupport.java#L112-L118 https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/java/org/gradle/api/plugins/jvm/internal/DefaultJvmPluginServices.java#L213-L236
should the plugin recommend the newest version outright even if it is not compatible with the newest version? What is the current behavior?
I believe that would be the least surprising result, as there are a variety of reasons why incompatibilities might occur (such as JavaEE => Jakarta, e.g Jetty 10 vs 11). A resolution rule can be applied to the report task to restrict expected incompatibilities until resolved.
Do you think that setting attributes, etc. can/should be done in this plugin or is that the responsibility of the consuming project? If the latter, what could we document for those who run into this problem?
Setting the attribute would be the responsibility of the plugin. You would only want to disable this auto-target-jvm behavior for the configurations that the plugin is resolving. During normal resolution in projects, this check is quite important. Otherwise, it would be very easy for a java 11 project to depend on java 17 bytecode -- which would break when actually running the code on a java 11 jvm.
I would presume it would make sense to add the attribute somewhere in createLatestConfiguration
https://github.com/jvandort/gradle-versions-plugin/blob/ce2e5c1d6a264f5e71deb871bf928ddb4190fd6f/gradle-versions-plugin/src/main/kotlin/com/github/benmanes/gradle/versions/updates/Resolver.kt#L89
We don't use spring, seeing the same behaviour so this might be a symptom of something else? Works without changing other deps. Only change is 0.44.0 -> 0.45.0.
Unfortunately using TARGET_JVM_VERSION_ATTRIBUTE, 0
did not work. See info log.
Incompatible because this component declares an API of a component compatible with Java 8 and the consumer needed a runtime of a component compatible with Java 0
@ben-manes Try Integer.MAX_VALUE
instead of 0. I got mixed up
@mikand13 Can you post your error here?
In that case, it doesn't seem to be applied.
Incompatible because this component declares a component compatible with Java 17 and the consumer needed a component compatible with Java 11
Interesting that it works with 0 but not max value. I don't think there's special logic for that but maybe there is. Try like 1000 or soemthing. If not i can investigate further
Nope, no luck with 1000 either.
Thanks all for looking into this. I downgraded to 0.44.0. Other than the noted one for log4j-core, I'm no longer getting the errors. I am still getting upgrade recommendations irrespective of my version range constraints.
Again adding a version alignment ComponentMetadataRule
to either my platform build or project convention plugin didn't make a difference. My hypothesis there was that maybe the BOM constraints weren't being considered by the plugin for the individual dependencies, but that the version alignment rule would transitively do the trick.
As for variants, it would be great to get update recommendations constrained by what's applicable to my project. I understand there could be a myriad of different variants to consider, but the Java target version is a very common case. Perhaps that could receive special treatment. Something like the following output would be really handy:
- org.springframework:spring-context [5.3.24 -> 5.3.25, 6.0.5 (1), 7.0.2 (2)]
(1) Incompatible because this component declares an API of a component compatible with Java 17 and the consumer needed a runtime of a component compatible with Java 11
(2) Incompatible because this component declares an API of a component compatible with Java 21 and the consumer needed a runtime of a component compatible with Java 11
Surely that's a separate feature request. For this issue I'm just hoping to get upgrade recommendations that are constrained by my explicitly specified platform and its imported BOMs, e.g. org.springframework:spring-context [5.3.24 -> 5.3.25]
instead of [5.3.24 -> 6.0.5]
.
My hypothesis there was that maybe the BOM constraints weren't being considered by the plugin for the individual dependencies.
The Spring dependency-management-plugin does not constrain the dynamic query (see #77). Otherwise it pinned the version so upgrades were not seen.
adding a version alignment ComponentMetadataRule to either my platform build or project convention plugin didn't make a difference
I'm unsure and am not familiar enough with all of the Gradle features, interactions, and implementation details. Generally since we use [Configuration.copyRecursive()](https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.Configuration.html#org.gradle.api.artifacts.Configuration:copyRecursive()), all of the resolution constraints are carried over and applied. Then additional can be added to the plugin's resolution strategy to customize the report for more restrictions. By letting Gradle handle it, I too would have expected the conventions to be honored. Offhand, we have seen cases where copyRecursive()
did not capture some settings and the Gradle team would fix these gaps when reported.
As for variants, it would be great to get update recommendations constrained by what's applicable to my project.
I suspect this would be hard because Gradle performs this logic and returns the resolved configuration, which we inspect to compare the dependency versions. We don't know why a version was rejected, but it is often captured in the info logs for users to debug from. The alternative is to do dependency resolution manually and then trying to emulate features like resolution strategies, constraints, variants, etc. from the maven pom and gradle module metadata. That's how Dependabot and similar work, which is useful but shallow. By letting Gradle handle it then you get more customized resolution and we have less duplication, but it does mean we are beholden to what the platform exposes.
I have not forgotten about this issue. I plan to take another look at this shortly
@ben-manes There was a slight issue with your patch. Your call to overwrite the TARGET_JVM_VERSION_ATTRIBUTE
in createLatestConfiguration
was made too early. And the call to addAttributes
later in the method overwrote the value.
Adding copy.attributes.attribute(TargetJvmVersion.TARGET_JVM_VERSION_ATTRIBUTE, Integer.MAX_VALUE)
on line 150 after the addAttributes(copy, configuration)
call (in addition to the change you made in getCurrentCoordinates
) solves the problem for me.
For reference, I used the following minimal reproducer based off of the reproducer given in the original post:
plugins {
id 'java-library'
id 'com.github.ben-manes.versions'
}
repositories {
mavenCentral()
}
testing {
suites {
integrationTest(JvmTestSuite) {
dependencies {
implementation(project)
}
}
}
}
dependencies {
implementation("org.springframework.boot:spring-boot:2.7.8")
}
oh thank you @jvandort!
I wasn't able to get the reproducer to work in a minimal project, as Gradle said implementation(project)
was invalid (bad parameter). While it would be nice to have a unit test, I'll release it as is. Thanks for all of the help everyone!
@ben-manes try implementation(project())
That works, but it does not fail?
------------------------------------------------------------
: Project Dependency Updates (report to plain text file)
------------------------------------------------------------
The following dependencies are using the latest milestone version:
- com.github.ben-manes:gradle-versions-plugin:0.45.0
The following dependencies have later milestone versions:
- org.springframework.boot:spring-boot [2.7.8 -> 3.0.2]
https://spring.io/projects/spring-boot
Gradle release-candidate updates:
- Gradle: [8.0.1: UP-TO-DATE]
Generated report file build/dependencyUpdates/report.txt
I thought the whole point of all of this was so it wouldn't fail?
I thought the reproducer would fail on the current release (v45) and pass on the next one (v46)?
Released in v46
I did notice a small quirk in v46 that if I set checkConstraints = true
then in Caffeine the resolution fails as below. If using v45 then it resolves fine and is listed as up-to-date.
Failed to determine the latest version for the following dependencies (use --info for details):
- com.beust:jcommander
1.82
The exception that is the cause of unresolved state: org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve com.beust:jcommander:+.
Required by:
project :
Caused by: org.gradle.internal.component.NoMatchingConfigurationSelectionException: No matching variant of com.beust:jcommander:1.82 was found. The consumer was configured to find attribute 'org.gradle.jvm.version' with value '2147483647' but:
- Variant 'apiElements' capability com.beust:jcommander:1.82:
- Incompatible because this component declares attribute 'org.gradle.jvm.version' with value '8' and the consumer needed attribute 'org.gradle.jvm.version' with value '2147483647'
- Variant 'runtimeElements' capability com.beust:jcommander:1.82:
- Incompatible because this component declares attribute 'org.gradle.jvm.version' with value '8' and the consumer needed attribute 'org.gradle.jvm.version' with value '2147483647'
I thought the reproducer would fail on the current release (v45) and pass on the next one (v46)?
What version of Java are you running gradle with? The reproducer does not specify a toolchain so if you're running with 17+ it should not error in v45
I'll see what I can figure out about caffeine
I've upgraded my reproducer to 0.46.0 (and Gradle 8.0.1):
I'm still getting "Failed to determine the latest version for the following dependencies" for log4j-core, and it's still recommending versions of Spring artifacts beyond my specified BOM constraints. Since this issue has been closed, should I file those as one or two separate issues?
@ianbrandt The recommending of artifacts beyond your BOM constraints should be a separate issue.
I am looking into the log4j issue currently
@ianbrandt The log4j problem was present on 0.44.0
before these changes. It is related to trying to resolve the buildscript classpath
configuration in the :subprojects
project, which has a log4j dependency constraint. Though since the :subprojects
project is empty with only a single subproject itself and no other configuration, it has no plugins.
You can either file a separate issue or add a repository to the :subprojects
project.
@ben-manes If you can provide a short quick reproducer for the caffeine issue you're seeing I can take a look at that
EDIT: Never mind, I got Caffeine to reproduce it.
Apologies for the delay. Things have been busy here recently with multiple major version and patch releases going out the door
EDIT: Never mind, I got Caffeine to reproduce it.
haha, that's great because I'm having a hard time making a minimal reproduction. I'm sure its something dumb I did in my build. 😄
I found the problem. Your root buildscript is doing dependency resolution (caused by dependencyUpdates
), but does not apply the jvm-ecosystem
plugin which is the plugin which adds the attribute schema compatibility rules which makes a value of Integer.MAX_VALUE
for TARGET_JVM_VERSION_ATTRIBUTE
compatible with 8
.
In general, any project which performs dependeny-resolution against JVM artifact must apply the jvm-ecosystem
plugin.
To resolve this, the versions plugin should apply the jvm-ecosystem
plugin to the current project.
For context, java-library
applies java
, which applies java-base
, which applies jvm-ecosystem
. Usually you do not need to worry about applying this plugin yourself. But, if you're plugin is setting attributes managed by the jvm-ecosystem plugin, of which TARGET_JVM_VERSION_ATTRIBUTE
is one of, the plugin must be applied.
beautiful and thank you! Can you open an issue?
Will do
The recommending of artifacts beyond your BOM constraints should be a separate issue.
Filed #755.
The log4j problem was present on
0.44.0
before these changes. It is related to trying to resolve the buildscriptclasspath
configuration in the:subprojects
project, which has a log4j dependency constraint. Though since the:subprojects
project is empty with only a single subproject itself and no other configuration, it has no plugins.You can either file a separate issue or add a repository to the
:subprojects
project.
Filed #756.
I'm defining my own platform that in turn imports the BOMs for Spring and Spring Boot. The project is still on Java 11, so for the time being I need to constrain and align my Spring and Spring Boot versions to be less than 6.x and 3.x respectively, as those require Java 17.
This may be user error, but I'm unable to get the Gradle Versions Plugin to honor my platform constraints. Reproducer here:
https://github.com/ianbrandt/gradle-versions-plugin-platform-constraints/tree/f5b4360c2c5725cce61fa1ea0ed7d307ef3e8ed0
Constraint checking enabled:
https://github.com/ianbrandt/gradle-versions-plugin-platform-constraints/blob/f5b4360c2c5725cce61fa1ea0ed7d307ef3e8ed0/build.gradle.kts#L26
My platform, with Spring BOM import, and version alignment via a
ComponentMetadataRule
(I'm not sure if the rule can be declared in the platform, or if it should be declared in my convention plugin, but I tried it both ways and it didn't seem to make a difference):https://github.com/ianbrandt/gradle-versions-plugin-platform-constraints/blob/f5b4360c2c5725cce61fa1ea0ed7d307ef3e8ed0/platforms/app-platform/build.gradle.kts
The version range constraints specified with strict version shorthand:
https://github.com/ianbrandt/gradle-versions-plugin-platform-constraints/blob/f5b4360c2c5725cce61fa1ea0ed7d307ef3e8ed0/gradle/libs.versions.toml#L13-L14
Toolchain declaration for Java 11 (via Kotlin):
https://github.com/ianbrandt/gradle-versions-plugin-platform-constraints/blob/main/build-logic/src/main/kotlin/com.ianbrandt.buildlogic.kotlin-project.gradle.kts#L13
With that I get the following running
dependencyUpdates
:When I run with
--info
I get:What I'm hoping for here is that the plugin recommends updates for the Spring BOMs I'm importing only if they're within a specified version range, and to not get the "Failed to determine the latest version for the following dependencies" errors for dependency versions that don't have variants compatible with my toolchain.
Am I doing something wrong, or is the issue limitations or bugs in the plugin?