search

GradleUp / shadow

Gradle plugin to create fat/uber JARs, apply file transforms, and relocate packages for applications and libraries. Gradle version of Maven's Shade plugin.
https://www.gradleup.com/shadow/
Apache License 2.0
3.73k stars 393 forks source link

Getting errors with Shadow 6.0.0 and Gradle version 6.6.1 (looks similar to older issues that were compatibility issues) #599

Closed StuAtGit closed 1 year ago

StuAtGit commented 4 years ago

Shadow Version

6.0.0

Gradle Version

6.6.1

Expected Behavior

I want to run the task: service:startShadowScripts

without errors (as required by the build pipeline I'm working with.

Actual Behavior

Fails and dumps an exception stack trace. The last specific message is:

Caused by: org.gradle.api.InvalidUserDataException: Value 'task ':service:shadowJar'' specified for property '$1' cannot be converted to a file.
        at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:71)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.lambda$execute$2(SkipEmptyWorkStep.java:78)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:78)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:34)

The full stack trace:

org.gradle.internal.execution.WorkValidationException: A problem was found with the configuration of task ':service:startShadowScripts' (type 'CreateStartScripts').
        at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:71)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.lambda$execute$2(SkipEmptyWorkStep.java:78)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:78)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:34)
        at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:39)
        at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:40)
        at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:28)
        at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:33)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:195)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:187)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:114)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:409)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:399)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:242)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:150)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:94)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:372)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:359)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:352)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:338)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
Caused by: org.gradle.api.InvalidUserDataException: Value 'task ':service:shadowJar'' specified for property '$1' cannot be converted to a file.
        at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:71)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.lambda$execute$2(SkipEmptyWorkStep.java:78)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:78)
        at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:34)
        at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:39)
        at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:40)
        at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:28)
        at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:33)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:195)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:187)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:114)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:409)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:399)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:242)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:150)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:94)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:372)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:359)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:352)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:338)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)

Gradle Build Script(s)

It's a messy thing worked on by several folks, none of us really experts in gradle. And I've been fighting with it for a bit.

Some information stripped for privacy:

buildscript {
    repositories {
        maven { url 'http://nexus.......com/content/repositories/snapshots' }
        maven { url 'http://nexus.......com/content/repositories/releases' }
    }

    dependencies {
        classpath 'com.github.jengelman.gradle.plugins:shadow:6.0.0'
    }
}
plugins {
    id 'org.springframework.boot' version '2.3.0.RELEASE'
    id 'io.spring.dependency-management' version '1.0.9.RELEASE'
    id 'java-library'
    id 'com.patdouble.awsecr' version '0.3.3' apply false
}

group = '.....'

allprojects {
    apply plugin: 'idea'
    apply plugin: 'eclipse'
    apply plugin: 'maven'
    repositories {
        jcenter()
        maven { url 'http://.....com/content/repositories/snapshots' }
        maven { url 'http://.....com/content/repositories/releases' }
        maven { url "https://repository.mulesoft.org/nexus/content/repositories/public/" }
    }
}

subprojects {
    apply plugin: 'java'
    apply plugin: 'maven'

    if( name == 'service' ) {
        apply plugin: 'application'
        mainClassName = 'My Main class'

        test {
            useJUnitPlatform()
                         ... environment ...
            //we want display the following test events
            testLogging {
                events "PASSED", "FAILED", "SKIPPED"
            }
        }
    } else {
        apply plugin: 'application'
        mainClassName = 'My main class'

        apply plugin: 'com.github.johnrengelman.shadow'
        apply plugin: '... required custom plugin.... '
        test {
            useTestNG() {
                useDefaultListeners = true
            }

            //we want display the following test events
            testLogging {
                events "PASSED", "FAILED", "SKIPPED"
            }
            onlyIf {
                project.hasProperty("enableSmokeTests")
            }
        }
    }
}

configurations{
    all {
        //Fix the following issue:
        //"org.apache.logging.log4j.LoggingException: log4j-slf4j-impl cannot be present with log4j-to-slf4j"
        exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
    }
}

project(':smoke') {
    dependencies {
        implementation group: 'com.amazon.redshift', name: 'redshift-jdbc42-no-awssdk', version: '1.2.41.1065'
        implementation group: 'org.jdbi', name: 'jdbi3-core', version: '3.13.0'
        implementation group: 'org.jdbi', name: 'jdbi3-sqlobject', version: '3.13.0'
        testImplementation 'com.google.code.gson:gson:2.8.6'
        testImplementation 'org.testng:testng:6.13'
    }
}

project(':service') {
    bootRun {
               environment ....
                ....
    }

    bootJar {
        mainClassName = '[My Main class]'
    }

    apply plugin: 'org.springframework.boot'
    apply plugin: 'io.spring.dependency-management'
    apply plugin: 'com.github.johnrengelman.shadow'
    apply plugin: '[some local custom plugin I'm required to use]'

    shadowJar {
        zip64 true
    }

    configurations{
        all {
            //Fix the following issue:
            //"org.apache.logging.log4j.LoggingException: log4j-slf4j-impl cannot be present with log4j-to-slf4j"
            exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
        }
    }

    dependencies {
        implementation 'org.springframework.boot:spring-boot-starter-web'
        implementation 'org.springframework.boot:spring-boot-starter-log4j2'
        implementation 'org.springframework.boot:spring-boot-starter-actuator'
        implementation 'org.springframework.retry:spring-retry:1.3.0'

        implementation 'commons-lang:commons-lang:2.6'
        implementation 'com.amazonaws:aws-java-sdk-s3:1.11.787'
        implementation 'com.amazonaws:aws-java-sdk-kinesis:1.11.804'
        implementation 'com.fasterxml.jackson.module:jackson-module-jaxb-annotations'
        implementation 'com.google.code.gson:gson:2.8.6'

        implementation 'io.springfox:springfox-swagger2:2.9.2'
        implementation 'io.springfox:springfox-swagger-ui:2.9.2'

        implementation('local blob of "helpful" packages') {
            exclude group: 'org.apache.logging.log4j', module: 'log4j-api'
            exclude group: 'org.apache.logging.log4j', module: 'log4j-core'
            exclude group: 'org.apache.logging.log4j', module: 'log4j-jcl'
            exclude group: 'org.slf4j', module: 'slf4j-nop'

            // XML Bind
            exclude group: 'javax.ws.rs', module: 'javax.ws.rs-api'
            exclude group: 'javax.xml.bind', module: 'jaxb-api'

            // Jersey
            exclude group: 'org.glassfish.jersey.containers', module: 'jersey-container-grizzly2-http'
            exclude group: 'org.glassfish.jersey.inject', module: 'jersey-hk2'
            exclude group: 'org.glassfish.jersey.media', module: 'jersey-media-json-jackson'
            exclude group: 'org.glassfish.jersey.core', module: 'jersey-client'
            exclude group: 'org.glassfish.jersey.test-framework.providers', module: 'jersey-test-framework-provider-grizzly2'
            exclude group: 'org.glassfish.jersey.test-framework.providers', module: 'jersey-test-framework-provider-inmemory'

            // Grizzly
            exclude group: 'org.glassfish.grizzly', module: 'grizzly-http-server'
            exclude group: 'org.glassfish.grizzly', module: 'grizzly-framework'
            exclude group: 'org.glassfish.grizzly', module: 'grizzly-websockets'

            // Jackson
            exclude group: 'com.fasterxml.jackson.core', module: 'jackson-core'
            exclude group: 'com.fasterxml.jackson.core', module: 'jackson-databind'
            exclude group: 'com.fasterxml.jackson.core', module: 'jackson-annotations'
            exclude group: 'com.fasterxml.jackson.module', module: 'jackson-module-jaxb-annotations'
            exclude group: 'com.fasterxml.jackson.module', module: 'jackson-module-parameter-modules'
            exclude group: 'com.fasterxml.jackson.datatype', module: 'jackson-datatype-jdk8'
            exclude group: 'com.fasterxml.jackson.datatype', module: 'jackson-datatype-jsr310'
            exclude group: 'com.fasterxml.jackson.dataformat', module: 'jackson-dataformat-cbor'

            // Jedis
            exclude group: 'redis.clients', module: 'jedis'

            // AWS
            exclude group: 'com.amazonaws', module: 'amazon-kinesis-producer'
            exclude group: 'com.amazonaws', module: 'amazon-kinesis-client'
            exclude group: 'com.amazonaws', module: 'amazon-elasticache-cluster-client'
            exclude group: 'com.amazonaws', module: 'amazon-dax-client'
            exclude group: 'com.amazonaws', module: 'aws-java-sdk-dynamodb'
            exclude group: 'com.amazonaws', module: 'aws-java-sdk-dax'
            exclude group: 'com.amazonaws', module: 'aws-java-sdk-s3'
            exclude group: 'org.eclipse.paho', module: 'org.eclipse.paho.client.mqttv3'
            exclude group: 'com.amazonaws', module: 'aws-iot-device-sdk-java'
            exclude group: 'com.amazonaws', module: 'aws-java-sdk-sts'
            exclude group: 'com.amazonaws', module: 'aws-java-sdk-sqs'

            // Swagger
            exclude group: 'io.swagger', module: 'swagger-jersey2-jaxrs'
            exclude group: 'org.webjars', module: 'swagger-ui'
        }
        testImplementation('org.springframework.boot:spring-boot-starter-test') {
            exclude group: 'org.junit.vintage', module: 'junit-vintage-engine'
        }
        testImplementation 'io.projectreactor:reactor-test'
    }
}

Content of Shadow JAR (jar tf <jar file> - post link to GIST if too long)

So, this would be huge, and pain to clean out everything I'm supposed to clean out of it. But, a couple things to note:

The problem seems isolated to that task, and seems to be some mistaken use of the gradle interface. I've filed a bug with gradle for them to print out a more user-friendly error: https://github.com/gradle/gradle/issues/14583

StuAtGit commented 4 years ago

If it helps, this is my gradle properties file:

distributionUrl=https\://services.gradle.org/distributions/gradle-6.6.1-all.zip
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStorePath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME

And this is the command I'm running that fails:

 ./gradlew --stacktrace service:startShadowScripts
StuAtGit commented 4 years ago

If it helps, I started fiddling around with debugging the gradle task, and I reached the point where I could extract this message out of a stack trace:

A problem was found with the configuration of task ':service:startShadowScripts' (type 'CreateStartScripts').

To be honest, though, not really sure what I'm looking at here, or what the problem with the configuration is...

marwanghalib commented 3 years ago

Looking at a similar issue: Caused by: org.gradle.api.InvalidUserDataException: Value 'directory '/xxxx/xxxx/xxxx/xxx' specified for property '$1' cannot be converted to a file.

Any updates on this issue?

johnrengelman commented 1 year ago

Closing this as we've moved up to Gradle 8 now.