Closed ianbrandt closed 3 months ago
For some reason it wants the sourcesElements
variant, but the authors decided not to publish that in their gradle module metadata. For example in my project's module metadata,
"name": "sourcesElements",
"attributes": {
"org.gradle.category": "documentation",
"org.gradle.dependency.bundling": "external",
"org.gradle.docstype": "sources",
"org.gradle.usage": "java-runtime"
},
For some reason the dependency did not include that in their module metadata.
I guess Kotlin multi-platform needs the sources and since those are not available in the metadata, it fails? They are included on the repository, though. This is more of a Gradle team question, as module metadata is confusing and the Kotlin tends to do odd things that are inconsistent with the ecosystem. I'm not the one capable of debugging this knot.
I retested with Gradle 8.7 and Kotlin 2.0.0-RC1. No significant change.
Failed to determine the latest version for the following dependencies (use --info for details):
- com.google.devtools.ksp:symbol-processing-api
2.0.0-RC1-1.0.20
- com.squareup:kotlinpoet-ksp
1.16.0
- io.kotest:kotest-assertions-shared
5.8.1
- org.jetbrains.kotlinx:kotlin-deeplearning-api
0.5.2
- org.jetbrains.kotlinx:kotlin-deeplearning-tensorflow
0.5.2
@ben-manes, Do you have a Gradle contact you'd typically reach out to for the sort of help you mention above? I can reference this in the #eap channel of the Kotlin Slack (they are asking for 2.0.0 RC testing feedback there), but I'm wondering if this isn't more of a Gradle than Kotlin issue.
I don't have one. I have simply been a nuisance for them since early years so interacted with many on their team. If one of their devs feels generous then they'll step in and help.
Just FYI, I retested with Kotlin 2.0.0 final. No significant change.
Reproducer: https://github.com/sdkotlin/sd-kotlin-talks/tree/dd8faa86c2ed99c514343862b68d40d3353ede61.
--info
...I posted a request for help in the Gradle Slack: https://gradle-community.slack.com/archives/CA745PZHN/p1716403012668999.
For some reason it wants the sourcesElements variant
Who is "it"?
For example in my project's module metadata,
That's because your use withSourcesJar()
which not only adds the respective sources jar task and publishes the resulting artifact but also models the proper variant so that it could be retrieved with variant aware resolution.
For some reason the dependency did not include that in their module metadata.
Because - just like you do it in this project too - they manually create the sources jar task and explicitly add the artifact to the publication instead of properly modeling it by simply using withSourcesJar()
. Imho that is a build / publishing bug that should be reported and fixed. It is most often historically grown when it was added before withSourcesJar()
existed and not replaced, or following some outdated blog posts or similar.
I guess Kotlin multi-platform needs the sources and since those are not available in the metadata, it fails?
I don't think so, especially as they are not published by the Kotlin plugin, but by the project build configuration, either not, the good way, or the bad way. If kmp would need it, it probably would also make sure to publish it?
Looking at :ksp-builder-generator:processor:resolvableConfigurations
output there are two configurations that request the sources which are dependencySources
and testDependencySources
. They extends compileClasspath
and testCompileClasspath
respectively with the attributes set to get the sources variants. I guess those configurations are not resolved directly, but only through a lenient artifact view. It would most probably have been a better idea to not add them as configurations but to just use an artifact view with variant reselection and lenient instead of adding a configuration that is more or less expected to only be partially resolvable.
Also read this commit message for example: https://github.com/JetBrains/kotlin/commit/73b290a94831a1aee547c99ab319fafc311864d7 🙄
Here the configuration is created: https://github.com/JetBrains/kotlin/blob/v2.0.0/libraries/tools/kotlin-gradle-plugin/src/common/kotlin/org/jetbrains/kotlin/gradle/plugin/sources/KotlinSourceSetFactory.kt#L135
This is the commit that added it: https://github.com/JetBrains/kotlin/commit/c979e4e7be88c61828c45e3f982522dc13293102 It is used by IDE sync but adding it during sync seems to be too late and so they moved it to the Kotlin plugin 2.0.0.
I guess it should either be moved to some init script used during sync or using some artifact view instead or similar but not adding such an intentionally only partly resolvable configuration. Or at least just add it during IDE sync which can be determined by system properties.
Imho this is a bug in the KGP 2.0.0 and needs to be fixed in a 2.0.1 at the earliest convenient time.
Thanks @Vampire!
I can only lightly reply to a few of your comments.
For some reason it wants the sourcesElements variant Who is "it"?
I was probably thinking that from the original error message,
- Other compatible attribute:
- Doesn't say anything about the documentation type (required sources)
Because - just like you do it in this project too - they manually create the sources jar task and explicitly add the artifact to the publication instead of properly modeling it by simply using withSourcesJar()... It is most often historically grown when it was added before withSourcesJar() existed and not replaced, or following some outdated blog posts or similar.
Yep, this project's build system was written for Gradle 1.3, published to central, and lightly updated across releases. No surprise ours would be pretty ancient and wrong headed! I try to keep caffeine's up to date as my primary project when I have free time. I agree we should do a clean up, as should they.
Imho this is a bug in the KGP 2.0.0 and needs to be fixed in a 2.0.1 at the earliest convenient time.
Since module metadata bugs happen in the wild, do you have any recommendations around our resolution to be more lenient? The non-module resolution via pom metadata is usually what most developers expect and all that Dependabot/Renovate support. I am sure modules are far better to resolve, but it would be nice to fallback to ignore them. I believe that is on a repository content basis, whereas for our lenient configuration resolution it would be nice to disable it on the copied configuration (e.g. as a fallback). I can try to fix my own mistakes but no longer chase down other projects to help fix theirs (which I am always grateful that you continue to do).
Since module metadata bugs happen in the wild
As I said, I don't think it is a bug in the module metadata. It is not mandatory to publish sources (on MavenCentral yes, but not on other repos). From the PoV of this variant-aware sources resolution, it is like not having sources published if the artifact is there but the variant is missing and having sources not published is a valid situation.
I still think it is a bug in the Kotlin Gradle plugin 2.0.0, that these configurations that are known to be only partly resolvable are added at all. Especially as it is only for IDE sync.
do you have any recommendations around our resolution to be more lenient?
In this case either say "not my problem" and make JetBrains fix it, or exclude those bad configurations hard-coded as work-around.
Or maybe better offer an option for consumers of your plugin to configure which configurations to ignore, then other similar cases can simple be worked-around by configuration.
It would probably also be very helpful also in other cases if you log the copied configuration name in the error message, that it is immediately clear where the problem is.
Of course you could do lenient resolution using an artifact view. But I'm not sure whether you should. I think in most cases when it fails, you have some bug like not copying the attributes properly and so on which were there in the past. And lenient resolution would just hide that, producing a result that might be wrong but appears to be sensible. Just like if you ask an AI tool to do something for you. :-D
The non-module resolution via pom metadata is usually what most developers expect
I don't agree actually. And if you would do, you would just make your results worse. And to do it you probably would have to manually rewrite the logic. You can of course configure the repository to ignore the Gradle Module Metadata and thus only use the POM, but that would mean you just loose the information and variants from the GMM and instead just have a few variants derived from the POM, resulting in even more such resolution errors. You would have to stop copying the correct attributes to the copied configuration and instead set the attributes of the main artifact. Which also can be problematic as different ecosystems might require different attributes, like the target platform for KMP.
and all that Dependabot/Renovate support.
That might be true.
You could also add refreshVersions
to the list as it also does not use the resolution engine but directly asks the Maven repository metadata which makes it much faster than this plugin, but only support explicitly supported repository types and produces less correct results.
That's one of the reasons I prefer your plugin over any alternative for checking updates. It just is much better, much more correct, and much more reliable. I usually combine it with refreshVersions
to get the comments with all possible update versions added. So please keep the quality up by doing it right and don't let you be dragged down by worse implementations.
I am sure modules are far better to resolve, but it would be nice to fallback to ignore them.
As I said, I don't agree, especially if the investigated build relies on variant resolution. You only get a worse result and miss potential bugs. I would still recommend to add the copied configuration name to the error and provide an option to exclude configurations and nothing more.
We do in fact allow skipping configurations!
I also prefer allowing gradle to resolve and combine with dependabot for its added security analysis.
I mentioned a fallback because I don’t like blocking users who are more best effort minded, so warning while not blocking them from getting their job done is my default preference. However it’s not a clean approach in the apis and I certainly wouldn’t add it as a massive hack, so it’s unlikely to ever be introduced.
We do in fact allow skipping configurations!
Ah, great, forgot about that.
But I still think outputting the original configuration name in error messages would be helpful or is that there to and I just overlooked it?
In this case I've could have seen it is all in the sources configurations and decide to just filter those as sources might not be available.
I mentioned a fallback because I don’t like blocking users who are more best effort minded, so warning while not blocking them from getting their job done is my default preference.
But logging bugs as warnings tremendously reduces the amount of actual bugs reported as from my experience many people tend to ignore warnings, as they are only warnings. :-/
As I said, in my opinion any resolution error there is a bug somewhere that should be fixed, including the possibility that it is a build bug that the configuration is analyzed and not filtered.
It is your baby of course, so your decision. I just offer my opinion for consideration. :-) So you might use a lenient artifact view to suppress missing variants. Or you could also make it configurable which configurations to treat lenient. But it might then be not possible to find the resolution problem to log it. But you already catch the resolution errors and log them, don't you? So you could just decide to not fail the task and just log them as warnings instead. But as I said imho the correct solution here would be to have the original configuration logged in the error message and the build to filter them it configure them to be treated leniently. You could also add a message hinting at these possibilities.
We don’t fail the task. It should log and skip the configuration if an unexpected failure.
Unfortunately the result aggregation doesn’t include the configurations the dependency was resolved on, so this report-level logging is ignorant. It would be nice to capture in the positive case as a version may vary across configurations and be non-obvious to track down.
I had imagined more like a flag option —lenient
to force awareness. I’m not active here so more pondering than action.
I created https://youtrack.jetbrains.com/issue/KT-68447 to hopefully get this fixed or improved in KGP
Tested with 2.0.20-Beta1, which purportedly has a fix for KT-68447. Now I get:
Failed to determine the latest version for the following dependencies (use --info for details):
- org.jetbrains.kotlin:kotlin-build-tools-impl
The exception that is the cause of unresolved state: org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find org.jetbrains.kotlin:kotlin-build-tools-impl:2.0.20-Beta1.
Required by:
project :ksp-builder-generator:processor
- org.jetbrains.kotlin:kotlin-reflect
2.0.0
The exception that is the cause of unresolved state: org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find org.jetbrains.kotlin:kotlin-reflect:2.0.20-Beta1.
Required by:
project :ksp-builder-generator:api:annotations
- org.jetbrains.kotlin:kotlin-stdlib
The exception that is the cause of unresolved state: org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find org.jetbrains.kotlin:kotlin-stdlib:2.0.20-Beta1.
Required by:
project :ksp-builder-generator:api:annotations
Reproducer: https://github.com/sdkotlin/sd-kotlin-talks/tree/a0ce4c6267cacaf1e955dfffb3a372e296d68c13.
Because you reject beta versions: https://github.com/sdkotlin/sd-kotlin-talks/blob/a0ce4c6267cacaf1e955dfffb3a372e296d68c13/build.gradle.kts#L55-L57
Ah, yes. Sorry, I wasn't thinking and forgot about the component selection filtering. I've updated that, and all looks good now. Thanks!
https://github.com/sdkotlin/sd-kotlin-talks/tree/cd24248817912216163f2dd6ffa0a65d061702ef
Confirmed as fixed upstream
When I upgrade my project to Kotlin 2.0.0-Beta4, I get the following from
dependencyUpdates
:Plugin version 0.51.0. Reproduces with Gradle 8.6 and 8.7-rc1.
Reproducer: https://github.com/sdkotlin/sd-kotlin-talks/tree/2363d9a0686164707d1256837cd343df098e51e1
I don't get the errors with Kotlin 1.9.22: https://github.com/sdkotlin/sd-kotlin-talks/tree/a412bc0b6c80436a31dad58f51608c1cd70da170
Here is one of the
--info
details (the rest appear to be similar):