gradle-nexus / publish-plugin

Gradle plugin for publishing to Nexus repositories
Apache License 2.0
404 stars 29 forks source link

Task ':publishXXX' uses this output of task ':signPluginMavenPublication' without declaring an explicit or implicit dependency - error in Gradle 8 #208

Open JavierSegoviaCordoba opened 1 year ago

JavierSegoviaCordoba commented 1 year ago

I have checked that the plugin should be compatible with Gradle 8 but I found this issue.

Reason: Task ':semver-project-gradle-plugin:publishJavaPublicationToSonatypeRepository' uses this output of task ':semver-project-gradle-plugin:signPluginMavenPublication' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

Possible solutions:

  1. Declare task ':semver-project-gradle-plugin:signPluginMavenPublication' as an input of ':semver-project-gradle-plugin:publishJavaPublicationToSonatypeRepository'.
  2. Declare an explicit dependency on ':semver-project-gradle-plugin:signPluginMavenPublication' from ':semver-project-gradle-plugin:publishJavaPublicationToSonatypeRepository' using Task#dependsOn.
  3. Declare an explicit dependency on ':semver-project-gradle-plugin:signPluginMavenPublication' from ':semver-project-gradle-plugin:publishJavaPublicationToSonatypeRepository' using Task#mustRunAfter.

Update XI 2023 (by the maintainer). The problem is not strictly related to this plugin and it seems to causes by the signing task with multiple publications. It is described by @Legion2 in this Gradle issue. There is also a workaround proposed, until it is fixed on the Gradle side.

szpak commented 1 year ago

Could you provide a way to reproduce that situation (sample project, command, etc)?

szpak commented 1 year ago

I assume you use the latest 1.3.0?

JavierSegoviaCordoba commented 1 year ago

1.2.0, AFAIK, there are no fixes on 1.3.0 regarding this, right?

szpak commented 1 year ago

1.2.0, AFAIK, there are no fixes on 1.3.0 regarding this, right?

Yes, it should be the same. However, we don't have a CI/CD in place, so there could be some local inconsistency :-/.

However, what Gradle version do you use? I didn't see that error with Gradle 6.4.1 used in our e2e tests (as I just checked the version, it would be good to update it). Is it enough to call that task? Or you do any validation task? Please provide a way to reproduce the aforementioned warning (error?).

JavierSegoviaCordoba commented 1 year ago

I started to see it on 8.0.2 (probably 8+). It is happening with tons of plugins, not only this one. And it started to happen on all plugins on Gradle 8+.

I am fixing it by myself in my DSL plugin (not tested yet) with

val signingTasks: TaskCollection<Sign> = tasks.withType<Sign>()
tasks.withType<PublishToMavenRepository>().configureEach { task ->
    task.mustRunAfter(signingTasks)
}

But I am not sure if it is a good approach as withType<Sign>() fetch eagerly all Sign tasks.

szpak commented 1 year ago

Gradle 8 compatibility

For some reasons, I originally perceived it as "Java 8 compatibility"... That's why I asked about your Gradle version...

u-ways commented 1 year ago

I've upgraded to latest (1.3.0) and the issue still persists, so I assume this is unresolved as of current. Any updates on this?

nedtwigg commented 1 year ago

I am also seeing

Build service 'stagingRepositoryUrlRegistry' is being used by task ':initializeSonatypeStagingRepository' without the corresponding declaration via 'Task#usesService'. This will fail with an error in Gradle 8.0. Declare the association between the task and the build service using 'Task#usesService'. Consult the upgrading guide for further information:

szpak commented 1 year ago

Reason: Task ':semver-project-gradle-plugin:publishJavaPublicationToSonatypeRepository' uses this output of task ':semver-project-gradle-plugin:signPluginMavenPublication' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

@JavierSegoviaCordoba I tried to reproduce the problem in compatibility tests with Wiremock, but without success. I even upgraded our e2e projects to Gradle 8.1.1 and they are still publishing fine (also the multimodule project in Kotlin). Have you by any chance been investigating what in the application causes the aforementioned error (or warning in 7.x)?

Preferably, maybe you could modify this e2e project (or if a multimodule structure is required that one)? It would make it easier to fix it in the plugin itself.

YarnSphere commented 1 year ago

Unrelated to the above but also related to Gradle 8 compatibility (at least with Gradle 8.1.1):

image

The repositories function is now clashing with Project.repositories from the Kotlin DSL. For now it seems to still compile, but it does show an error in the IDE. Doing something like the following fixes it (since thankfully the Project.repositories argument has a different name):

image

It might make sense to introduce an alias to repositories such as nexusRepositories and perhaps deprecate the former.

If I'm missing something obvious and there's a less hacky workaround please do let me know!

I tried going over the Gradle release notes to check if this change in behaviour is mentioned anywhere but couldn't find anything to reference. I'd appreciate if someone could find a reference as to why this is now happening.

Also, please let me know if you'd prefer that I create a new issue to track this.

YarnSphere commented 1 year ago

Regarding the original issue, I just attempted a publish with Gradle 8.1.1 and got the following error:

FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task ':my-project:signJvmPublication' (type 'Sign').
  - Gradle detected a problem with the following location: '/.../my-project/build/libs/my-project-0.0.0-javadoc.jar.asc'.

    Reason: Task ':my-project:publishJsPublicationToSonatypeRepository' uses this output of task ':my-project:signJvmPublication' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

    Possible solutions:
      1. Declare task ':my-project:signJvmPublication' as an input of ':my-project:publishJsPublicationToSonatypeRepository'.
      2. Declare an explicit dependency on ':my-project:signJvmPublication' from ':my-project:publishJsPublicationToSonatypeRepository' using Task#dependsOn.
      3. Declare an explicit dependency on ':my-project:signJvmPublication' from ':my-project:publishJsPublicationToSonatypeRepository' using Task#mustRunAfter.

This is in a multi-module repository, the package in question is from a Kotlin multiplatform project. I'll compare it with your sample project to see if I can reproduce it there.

YarnSphere commented 1 year ago

@szpak, I didn't try to reproduce it with your example project, but I created a minimal reproduction with Kotlin multiplatform at: https://github.com/YarnSphere/nexus-publish-gradle-8

The issue appears when adding a Javadoc artifact to the publications and trying to sign them. In the example I create a Javadoc jar from Dokka's output and add it as an artifact to the publications.

Running the publishToMavenLocal task should reproduce the issue.

Hope this helps! Let me know if you need something else.

skydoves commented 1 year ago

I'm facing the same issue from my entire projects: https://github.com/skydoves/sandwich/actions/runs/4769237431/jobs/8479418913

YarnSphere commented 1 year ago

Regarding https://github.com/gradle-nexus/publish-plugin/issues/208#issuecomment-1520845369, where I say:

If I'm missing something obvious and there's a less hacky workaround please do let me know!

The less hacky workaround is simply:

nexusPublishing {
    this.repositories {
        sonatype()
    }
}

Which I find acceptable (I don't know how I originally missed this :sweat_smile:).

skydoves commented 1 year ago

@YarnSphere Thanks for sharing your solution. But the plugin has a fundamental issue with gradle 8.0.

Reason: Task ':sandwich-datasource:generateMetadataFileForReleasePublication' uses this output of task ':sandwich-datasource:androidSourcesJar' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

    Possible solutions:
      1. Declare task ':sandwich-datasource:androidSourcesJar' as an input of ':sandwich-datasource:generateMetadataFileForReleasePublication'.
      2. Declare an explicit dependency on ':sandwich-datasource:androidSourcesJar' from ':sandwich-datasource:generateMetadataFileForReleasePublication' using Task#dependsOn.
      3. Declare an explicit dependency on ':sandwich-datasource:androidSourcesJar' from ':sandwich-datasource:generateMetadataFileForReleasePublication' using Task#mustRunAfter.
skydoves commented 1 year ago

This plugin seems to work well with Gradle 8.0.

https://github.com/vanniktech/gradle-maven-publish-plugin

carlpoole commented 1 year ago

Update: See here for the solution I found for the issue I encountered in this comment https://github.com/gradle-nexus/publish-plugin/issues/208#issuecomment-1548869188 regarding publishing my Android lib.

Original comment Our project updated to Gradle 8 and I think we are blocked on the same issue

Reason: Task ':signReleasePublication' uses this output of task ':bundleReleaseAar' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

    Possible solutions:
      1. Declare task ':bundleReleaseAar' as an input of ':signReleasePublication'.
      2. Declare an explicit dependency on ':bundleReleaseAar' from ':signReleasePublication' using Task#dependsOn.
      3. Declare an explicit dependency on ':bundleReleaseAar' from ':signReleasePublication' using Task#mustRunAfter.

    Please refer to https://docs.gradle.org/8.0.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.

Update:

I was able to fix my build problem by adding dependsOn(':bundleReleaseAar') into a task I had in my publishing code

Removed this fix snippet because it was wrong and cut out my dependencies from the POM... oops.

szpak commented 1 year ago

@szpak, I didn't try to reproduce it with your example project, but I created a minimal reproduction with Kotlin multiplatform at: https://github.com/YarnSphere/nexus-publish-gradle-8

Thanks for the reproducer, however, it seems to not related with the GNPP. I commented out the whole nexusPublishing {} and even id("io.github.gradle-nexus.publish-plugin") version "1.3.0" in your project and it still fails:

$ gw clean publishToMavenLocal
Starting a Gradle Daemon, 1 incompatible and 1 stopped Daemons could not be reused, use --status for details

> Task :dokkaHtml
Initializing plugins
Dokka is performing: documentation for nexus-publish-example
Validity check
Creating documentation models
Transforming documentation model before merging
Merging documentation models
Transforming documentation model after merging
Creating pages
Transforming pages
Rendering
Running post-actions

> Task :signKotlinMultiplatformPublication FAILED

FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task ':signKotlinMultiplatformPublication' (type 'Sign').
  - Gradle detected a problem with the following location: '/tmp/nexus-publish-gradle-8/build/libs/nexus-publish-example-1.0-SNAPSHOT-javadoc.jar.asc'.

    Reason: Task ':publishJsPublicationToMavenLocal' uses this output of task ':signKotlinMultiplatformPublication' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

    Possible solutions:
      1. Declare task ':signKotlinMultiplatformPublication' as an input of ':publishJsPublicationToMavenLocal'.
      2. Declare an explicit dependency on ':signKotlinMultiplatformPublication' from ':publishJsPublicationToMavenLocal' using Task#dependsOn.
      3. Declare an explicit dependency on ':signKotlinMultiplatformPublication' from ':publishJsPublicationToMavenLocal' using Task#mustRunAfter.

    Please refer to https://docs.gradle.org/8.1.1/userguide/validation_problems.html#implicit_dependency for more details about this problem.

You should probably orchestrate the regular publishing with the signing.

szpak commented 1 year ago

@YarnSphere Thanks for sharing your solution. But the plugin has a fundamental issue with gradle 8.0.


Reason: Task ':sandwich-datasource:generateMetadataFileForReleasePublication' uses this output of task ':sandwich-datasource:androidSourcesJar' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.

@skydoves Neither generateMetadataFileForReleasePublication nor androidSourcesJar is created by this plugin. Are you really sure that GNPP "breaks it" and "has a fundamental issue with gradle 8.0"? Maybe, as it seems to be in the another mentioned here sample project, even without this plugin the build fails?

There was a real issue with deprecation in Gradle 8, but it was fixed in 1.2.0.

skydoves commented 1 year ago

I fixed my issues by adding a dependency like the one below:

tasks.named("generateMetadataFileForReleasePublication").configure { .. }
YarnSphere commented 1 year ago

Sorry for the delay in responding and thank you for looking into it, @szpak. I was under the misguided impression that GNPP provided the publishToMavenLocal task, but that is handled by the maven-publish plugin itself.

At least in regard to my reproduction, I agree with you in that this doesn't seem like GNPP's fault since the publishing tasks provided by GNPP fail in the same way that publishToMavenLocal does.

This still smells like a bug to me, though, I find it weird that we have to manually specify that a publishing task depends on the signing of its artifacts; this should, in my opinion, be implicit. Do you have any clue on which component would be responsible for this, in order to perhaps open a ticket in the correct repo? I'm not sure whether this would be an issue with the maven-publish plugin, kotlin(-multiplatform) plugin, or elsewhere.

szpak commented 1 year ago

This still smells like a bug to me, though, I find it weird that we have to manually specify that a publishing task depends on the signing of its artifacts; this should, in my opinion, be implicit. Do you have any clue on which component would be responsible for this, in order to perhaps open a ticket in the correct repo? I'm not sure whether this would be an issue with the maven-publish plugin, kotlin(-multiplatform) plugin, or elsewhere.

@YarnSphere I'm not sure, personally, I'm lagging behind the latest Gradle releases. Maybe it would be good to ask about that on the Gradle forum?

carlpoole commented 1 year ago

Just wanted to drop an update to my previous comment in case it helps anyone (https://github.com/gradle-nexus/publish-plugin/issues/208#issuecomment-1535022566).

My project is Android and I encountered my issue which seemed similar. The previous fix I mentioned wasn't quite right, but this was the ultimate solution I found for my Android library project. I added this publishing block in the android block of my build.gradle file and the problem went away entirely.

android {
    ...

    publishing {
        singleVariant("release")
    }
}

I found it here https://developer.android.com/reference/tools/gradle-api/7.1/com/android/build/api/dsl/Publishing and I think it might be related to the Android Gradle Plugin changes that coincided with Gradle 8 in this specific case https://developer.android.com/build/releases/gradle-plugin#8-0-0

Once this was added the publish script was able to locate components.release and the error mentioned previously did not occur.

szpak commented 1 year ago

Thanks @carlpoole for sharing your findings.

Two general questions (anyone here, please reply, if you can answer YES, to any of those questions):

  1. Has anybody here encountered the aforementioned problem with signing in a NON-Android project?
  2. Is in any project a workaround proposed a comment above did NOT help (or it couldn't be used for any reasons)?

I'm not an Android developer, so I would like to know if this is something I should put to the README to help people dealing with similar issues.

JavierSegoviaCordoba commented 1 year ago

I have found it in KMP, but I don't remember if it has android too.

YarnSphere commented 1 year ago

The reproduction that I posted above that we concluded is not GNPP's fault (even though the issue still occurs when publishing via a GNPP task) is not an Android project. It is Kotlin multiplatform, but it doesn't even have an Android target.

szpak commented 1 year ago

The reproduction that I posted above that we concluded is not GNPP's fault (even though the issue still occurs when publishing via a GNPP task) is not an Android project. It is Kotlin multiplatform, but it doesn't even have an Android target.

Right. So maybe multiple publishing variants is a key? @YarnSphere Could you test it with the latest Gradle 8.1.1 and maybe ask about that using the Gradle forum?

YarnSphere commented 1 year ago

Yeah, can do. I'll give it a try at reproducing it without Kotlin Multiplatform as well.

YarnSphere commented 1 year ago

It's worth noting as well, by the way, I don't think the Kotlin plugin officially supports Gradle 8 yet (https://kotlinlang.org/docs/whatsnew1820.html#gradle), so I wouldn't be surprised if this ends up being a problem with the Kotlin plugin. I'm curious if anyone here has managed to reproduce this in a non-Kotlin project, or at least in a non-Kotlin Multiplatform project.

Legion2 commented 11 months ago

Here is a reproducer where publishToMavenLocal does not fail. It uses the latest Kotlin plugin version (1.9.21) which supports gradle 8.

Legion2 commented 11 months ago

I investigated the problem further and I posted my findings here. It looks like a problem with the gradle signing plugin in combination with multiple publications which use the same input artifact.

szpak commented 11 months ago

Thanks @Legion2. I've updated the original issue report to mention your analysis and a workaround (as it might be useful for the other affected users).