ben-manes
commented
2 years ago What is the dependency coordinate?
Closed sschuberth closed 2 years ago
ben-manes
commented
2 years ago What is the dependency coordinate?
sschuberth
commented
2 years ago I'm using implementation("com.sksamuel.hoplite:hoplite-core:1.4.15"), and https://github.com/sksamuel/hoplite/releases/tag/v1.4.16 is not found, although it's at https://mvnrepository.com/artifact/com.sksamuel.hoplite/hoplite-core/1.4.16.
ben-manes
commented
2 years ago Maybe it is a resolution rule in your build? I was able to modify the Groovy example build to resolve 1.4.16.
sschuberth
commented
2 years ago Hmm, can it be something Kotlin DSL specific? I commented out
but still the hoplite update does not show up. I'll try to investigate.
sschuberth
commented
2 years ago I just realized hoplite is also not part of the "The following dependencies are using the latest milestone version" output, so the dependency does not seem to be found at all...
Edit: ./gradlew :model:dependencies does list it.
ben-manes
commented
2 years ago yeah, the kotlin sample works fine.
sschuberth
commented
2 years ago yeah, the kotlin sample works fine.
Thanks for checking. Then it must be something specific to our project layout...
sschuberth
commented
2 years ago Hmm. The dependencyUpdates task is not available in the subprojects, only in the root of our multiproject. According to
Alternatively, you can run the task separately in each subproject to generate separate reports for each subproject.
I thought it should work. I'll look further.
ben-manes
commented
2 years ago If you apply it to a subproject then it will run there, but when I did that I don't see hoplite in the list
> Task :model:dependencyUpdates
------------------------------------------------------------
:model Project Dependency Updates (report to plain text file)
------------------------------------------------------------
The following dependencies are using the latest milestone version:
- com.fasterxml.jackson.core:jackson-databind:2.13.1
- com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.13.1
- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.1
- io.gitlab.arturbosch.detekt:detekt-formatting:1.19.0
- io.kotest:kotest-assertions-core:5.0.3
- io.kotest:kotest-runner-junit5:5.0.3
- io.mockk:mockk:1.12.2
- org.jetbrains.dokka:dokka-base:1.6.10
- org.jetbrains.kotlin:kotlin-scripting-compiler-embeddable:1.6.10
The following dependencies have later milestone versions:
- com.networknt:json-schema-validator [1.0.64 -> 1.0.65]
https://github.com/networknt/json-schema-validator
Gradle release-candidate updates:
- Gradle: [7.3.3: UP-TO-DATE]If you apply it to a subproject then it will run there
Oh, that's unexpected. I was reading "Alternatively, you can run the task separately in each subproject to generate separate reports for each subproject" from the README as if appliying the plugin to only the root project should make the dependencyUpdates task also available in the subprojects.
but when I did that I don't see hoplite in the list
Even more interestingly: When applying the plugin only to the root project, the update for com.networknt:json-schema-validator is correctly detected although it's only used in the model subproject. The hoplite-core dependency, though, it not found at all, and also no update for it.
ben-manes
commented
2 years ago Some configurations can't be resolved, now by Gradle's design (resolvable and consumable configurations). Since this is a quirky new restriction that is not always set correctly, the most robust approach was to attempt and skip on an exception. In the --info logs it notes Skipping configuration :model:implementation with an exception. It would seem that some build logic is triggering a mutation on a different configuration, which leads to an error from preventIllegalMutation. Since we use a copy to mutate, I don't think this is the plugin's fault and some surprising interaction of an overly broad build rule.
sschuberth
commented
2 years ago Hmm. But a simple
./gradlew --info tasks | grep "Skipping configuration"or
./gradlew --info allDependencies | grep "Skipping configuration"does not show anything, whereas
./gradlew --info dependencyUpdates | grep "Skipping configuration"shows
Skipping configuration :cli:funTestCompileClasspath
Skipping configuration :cli:funTestRuntimeClasspath
Skipping configuration :cli:implementation
Skipping configuration :evaluator:funTestCompileClasspath
Skipping configuration :evaluator:funTestRuntimeClasspath
Skipping configuration :evaluator:implementation
Skipping configuration :model:funTestCompileClasspath
Skipping configuration :model:funTestRuntimeClasspath
Skipping configuration :model:implementation
Skipping configuration :utils:common-utils:funTestCompileClasspath
Skipping configuration :utils:common-utils:funTestRuntimeClasspath
Skipping configuration :utils:scripting-utils:funTestCompileClasspath
Skipping configuration :utils:scripting-utils:funTestRuntimeClasspath
Skipping configuration :utils:scripting-utils:implementationSo to me, it does seem to be related to the plugin after all...
sschuberth
commented
2 years ago Also, the exception message says
Could not resolve all dependencies for configuration ':model:implementationCopy'Note the "Copy" suffix. Is that the copy that the version plugin creates? And if it only complains about :model:implementationCopy, I'd assume the resolution of the original :model:implementation went fine. So the question is: Why can the original be resolved, but the copy not?
ben-manes
commented
2 years ago The plugin copies the configuration and resolves it. Sometimes others will hard code a configuration name to whitelist resolution behavior, which is brittle. There seems to be some assumption in your build, either direct or from a plugin. Why a totally different configuration name is in the exception leads me to think that something else is fragile.
Cannot change dependencies of dependency configuration ':clients:clearly-defined:api' after it has been included in dependency resolution.
The logging from the plugin is the source configuration, which the Configuration#copy() names itself from, and the error is the exception from resolving that.
sschuberth
commented
2 years ago Same thing with the "Cannot change dependencies of dependency configuration" exception, I'm only getting that with
./gradlew --info dependencyUpdates | grep "Cannot change dependencies of dependency configuration"but not with
./gradlew --info allDependencies | grep "Cannot change dependencies of dependency configuration"As another indicator that the problem might be in this plugin after all, the refreshVersions plugin properly detects the update for hoplite-core in the same project.
ben-manes
commented
2 years ago We resolve a copy which the other tasks don’t. So while this plugin encounters this issue, that doesn’t mean it’s our bug. If something is breaking core Gradle apis then what are we supposed to fix? I don’t know the root cause in your case and don’t yet see what this plugin can do to fix it.
refreshVersions takes the other approach of reinventing dependency resolution. That means they have to write code for each type of repository (mvn, ivy, file, s3, etc), handle secure proxies, reinvent resolution rules, etc. They of course don’t, so you miss functionality and slowly diverge from the build’s expectations. For example their support for dependencyResolutionManagement was custom and delayed, while we got the feature without changes. The benefits of leveraging the platform is also a liability, so we’re at the mercy of what the underlying system does. If another plugin breaks that then we’re impacted and only sometimes can hack around their abuses (e.g. kotlin plugin).
Unfortunately I don’t know how to quickly debug this problem further to identify the root problem.
sschuberth
commented
2 years ago So while this plugin encounters this issue, that doesn’t mean it’s our bug.
Agreed. I was just trying to collect hints.
If something is breaking core Gradle apis then what are we supposed to fix?
Of course not 😺
refreshVersions takes the other approach of reinventing dependency resolution.
Yeah, I also realized that plugin has downsides. Which is why I'd prefer to stick with this one 😀
Unfortunately I don’t know how to quickly debug this problem further to identify the root problem.
That already is good feedback, thanks. If you're out of ideas, I probably don't even need to try 😆
ben-manes
commented
2 years ago What confuses me is I don't know what this means,
Cannot change dependencies of dependency configuration ':clients:clearly-defined:api' after it has been included in dependency resolution.
When we rewrite the configuration for resolution, we copy it, clear the dependencies, and write back only external dependencies with the + version. The project dependencies are dropped and I don't know what that error means or is referring to.
ben-manes
commented
2 years ago My work day ended so I dug into this.
It's probably worth noting that if you comment out the api projects in the model's build then it resolves fine. Or if I keep those project dependencies and remove the api libraries instead then it passes. However the implementationCopy only includes the external dependencies when printed out.
The stacktrace points to KotlinDependenciesManagement.configureStdlibDefaultDependency as trying to add dependencies at runtime. This is not configuration-oriented but by project, as it modifies other configurations that are found within the scope's hierarchy. This seems to imply then that since it sees the other project dependency it tries to modify all of those in the project's subtree, which are already resolved, and then causes an error. It doesn't properly handle Configuration#copy() may be called and resolved by a task, which is valid Gradle behavior (e.g. often by using a detatched configuration). The oversight here is to assume that all configurations are known by the end of the configuration phase, and trying to be excessively magical by auto-configuring dependencies instead of following Gradle's preference of explicitness.
In short, this seems to be a bug in the Kotlin plugin and requires that they fix their behavior to abide by the APIs. You could try reaching out to them to see if they can find a nice workaround in their logic.
sschuberth
commented
2 years ago My work day ended so I dug into this.
Thanks a lot for the investigation!
You could try reaching out to them to see if they can find a nice workaround in their logic.
sschuberth
commented
2 years ago Turns out there's already a related bug report at https://youtrack.jetbrains.com/issue/KT-41642 which contains a work-around that might also be applicable to the gradle-versions-plugin. Use:
configuration.withDependencies {
try { action() } catch(e: InvalidUserDataException) ...
}That workaround was a request for Jetbrains to use it, as withDependencies states Execute the given action before the configuration first participates in dependency resolution. This would allow them to add their stdlib dependency once before resolution and not try to mutate it afterwards. It is a more correct API for them to use, but does not help anyone affected by their mistake. They should have used defaultDependencies(action) which is meant exactly for their use-case.
The author's fix of disabling the default dependency might help you. When I do that then the project resolves correctly, but you'll have to manage this dependency yourself.
sschuberth
commented
2 years ago ben-manes
commented
2 years ago @natario1 is the one who suggested it in KT-41642. He did a great job finding a workaround for users and proposing a fix for the Kotlin team. It's too bad that they ignored his bug report.
Please add that to KT-50807. You found the other bug report with the answers, I just gave it a second reading and saw this extra tidbit of insight in it.
sschuberth
commented
2 years ago So, the work-around works for me, and in the end it indeed turned out to not be an issue in this plugin. Thus I'm closing this as there's nothing to fix
For visibility I'd like to document that updates to the https://github.com/sksamuel/hoplite library are not detected by this plugin. I've not yet investigate why that's the case, or whether the root cause is an already known issue.