SnykMergedDepsConf error when using Shadow plugin v6.0.0

[mapoulin]

node -v: v14.5.0 npm -v: 6.14.7 snyk -v: 1.366.0 gradle -v: 6.5.1

Command run: snyk test

Expected behaviour

When running with com.github.johnrengelman.shadow plugin version 5.2.0

✓ Tested 335 dependencies for known issues, no vulnerable paths found.

Actual behaviour

When running with com.github.johnrengelman.shadow plugin version 6.0.0

Gradle Error (short):
> Could not resolve all dependencies for configuration ':pleo-phobos-app:snykMergedDepsConf'.
   > Could not resolve project :pleo-phobos-rest.
      > The consumer was configured to find a component compatible with Java 8, packaged as a jar, as well as attribute 'artifactType' with value 'script-files-extensions'. However we cannot choose between the following variants of project :pleo-phobos-rest:

===== DEBUG INFORMATION START =====
gradle command: '/Users/marc/Documents/repositories/phobos/gradlew' snykResolvedDepsJson -q --build-file build.gradle --no-daemon -Dorg.gradle.parallel= -Dorg.gradle.console=plain -PonlySubProject=pleo-phobos-app -I /var/folders/sh/_49nr_710f1fnhkx19rsjxx40000gn/T/tmp-38647-9w5u3qHbdit7--init.gradle

------------------------------------------------------------
Gradle 6.5.1
------------------------------------------------------------

Build time:   2020-06-30 06:32:47 UTC
Revision:     66bc713f7169626a7f0134bf452abde51550ea0a

Kotlin:       1.3.72
Groovy:       2.5.11
Ant:          Apache Ant(TM) version 1.10.7 compiled on September 1 2019
JVM:          11.0.2 (Oracle Corporation 11.0.2+9)
OS:           Mac OS X 10.15.4 x86_64

>>> command: '/Users/marc/Documents/repositories/phobos/gradlew' snykResolvedDepsJson -q --build-file build.gradle --no-daemon -Dorg.gradle.parallel= -Dorg.gradle.console=plain -PonlySubProject=pleo-phobos-app -I /var/folders/sh/_49nr_710f1fnhkx19rsjxx40000gn/T/tmp-38647-9w5u3qHbdit7--init.gradle
>>> exit code: 1
>>> stdout:
SNYKECHO snykResolvedDepsJson task is executing via doLast
JSONATTRS {"org.gradle.usage":["kotlin-api","java-runtime","java-api"],"org.gradle.libraryelements":["jar"],"org.gradle.dependency.bundling":["external","embedded"],"org.gradle.category":["library","documentation"],"org.jetbrains.kotlin.platform.type":["jvm","common"],"org.jetbrains.kotlin.localToProject":["local to :pleo-phobos-app","local to :pleo-phobos-rest"],"artifactType":["script-files-extensions"],"org.gradle.jvm.version":["8"],"org.gradle.docstype":["javadoc","sources"]}
SNYKECHO processing project: pleo-phobos-app
SNYKECHO constructing merged configuration from [-api, -runtime, annotationProcessor, api, apiDependenciesMetadata, apiElements, archives, bootArchives, compile, compileClasspath, compileOnly, compileOnlyDependenciesMetadata, default, developmentOnly, implementation, implementationDependenciesMetadata, jacocoAgent, jacocoAnt, kotlinCompilerClasspath, kotlinCompilerPluginClasspath, kotlinNativeCompilerPluginClasspath, kotlinScriptDef, kotlinScriptDefExtensions, productionRuntimeClasspath, runtime, runtimeClasspath, runtimeElements, runtimeOnly, runtimeOnlyDependenciesMetadata, sourceArtifacts, testAnnotationProcessor, testApi, testApiDependenciesMetadata, testCompile, testCompileClasspath, testCompileOnly, testCompileOnlyDependenciesMetadata, testImplementation, testImplementationDependenciesMetadata, testKotlinScriptDef, testKotlinScriptDefExtensions, testRuntime, testRuntimeClasspath, testRuntimeOnly, testRuntimeOnlyDependenciesMetadata]
SNYKECHO resolving configuration snykMergedDepsConf

>>> stderr:

FAILURE: Build failed with an exception.

* Where:
Initialization script '/var/folders/sh/_49nr_710f1fnhkx19rsjxx40000gn/T/tmp-38647-9w5u3qHbdit7--init.gradle' line: 258

* What went wrong:
Execution failed for task ':snykResolvedDepsJson'.
> Could not resolve all dependencies for configuration ':pleo-phobos-app:snykMergedDepsConf'.
   > Could not resolve project :pleo-phobos-rest.
     Required by:
         project :pleo-phobos-app
      > The consumer was configured to find a component compatible with Java 8, packaged as a jar, as well as attribute 'artifactType' with value 'script-files-extensions'. However we cannot choose between the following variants of project :pleo-phobos-rest:
          - compile
          - default
          - runtime
          - testCompile
          - testRuntime
        All of them match the consumer attributes:
          - Variant 'compile' capability pleo-io.phobos:pleo-phobos-rest:5.2.0:
              - Unmatched attributes:
                  - Doesn't say anything about artifactType (required 'script-files-extensions')
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Provides attribute 'org.jetbrains.kotlin.localToProject' with value 'local to :pleo-phobos-rest' but the consumer didn't ask for it
                  - Provides attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm' but the consumer didn't ask for it
          - Variant 'default' capability pleo-io.phobos:pleo-phobos-rest:5.2.0:
              - Unmatched attributes:
                  - Doesn't say anything about artifactType (required 'script-files-extensions')
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Provides attribute 'org.jetbrains.kotlin.localToProject' with value 'local to :pleo-phobos-rest' but the consumer didn't ask for it
                  - Provides attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm' but the consumer didn't ask for it
          - Variant 'runtime' capability pleo-io.phobos:pleo-phobos-rest:5.2.0:
              - Unmatched attributes:
                  - Doesn't say anything about artifactType (required 'script-files-extensions')
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Provides attribute 'org.jetbrains.kotlin.localToProject' with value 'local to :pleo-phobos-rest' but the consumer didn't ask for it
                  - Provides attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm' but the consumer didn't ask for it
          - Variant 'testCompile' capability pleo-io.phobos:pleo-phobos-rest:5.2.0:
              - Unmatched attributes:
                  - Doesn't say anything about artifactType (required 'script-files-extensions')
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Provides attribute 'org.jetbrains.kotlin.localToProject' with value 'local to :pleo-phobos-rest' but the consumer didn't ask for it
                  - Provides attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm' but the consumer didn't ask for it
          - Variant 'testRuntime' capability pleo-io.phobos:pleo-phobos-rest:5.2.0:
              - Unmatched attributes:
                  - Doesn't say anything about artifactType (required 'script-files-extensions')
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Provides attribute 'org.jetbrains.kotlin.localToProject' with value 'local to :pleo-phobos-rest' but the consumer didn't ask for it
                  - Provides attribute 'org.jetbrains.kotlin.platform.type' with value 'jvm' but the consumer didn't ask for it
        The following variants were also considered but didn't match the requested attributes:
          - Variant 'apiElements' capability pleo-io.phobos:pleo-phobos-rest:5.2.0 declares a component, packaged as a jar:
              - Incompatible because this component declares a component compatible with Java 11 and the consumer needed a component compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about artifactType (required 'script-files-extensions')
          - Variant 'runtimeElements' capability pleo-io.phobos:pleo-phobos-rest:5.2.0 declares a component, packaged as a jar:
              - Incompatible because this component declares a component compatible with Java 11 and the consumer needed a component compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about artifactType (required 'script-files-extensions')

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 10s

===== DEBUG INFORMATION END =====

Error running Gradle dependency analysis.

Please ensure you are calling the `snyk` command with correct arguments.
If the problem persists, contact support@snyk.io, providing the full error
message from above, starting with ===== DEBUG INFORMATION START =====.

Steps to reproduce

plugins {
    id "application"
    // ugrade shadow to 6.0.0 will make snyk explode...
    id "com.github.johnrengelman.shadow" version "5.2.0"
}

---

If applicable, please append the --debug flag on your command and include the output here **ensuring to remove any sensitive/personal details or tokens.

[lili2311]

👋 Have you seen https://github.com/snyk/snyk-gradle-plugin/issues/130? Could you try the suggestions from there?

[lili2311]

@mapoulin did the suggestion resolve your issue?

[mapoulin]

Possibly, for now we are refraining from upgrading shadow and remain on the 5.2.0 version.

[lili2311]

@mapoulin are you able at all to share a link to a project that uses the plugin or provide an example that shows the problem so we can look into this further to understand how to work with shadow 5.2.0?

[shanman190]

@lili2311, I've attempted to reproduce a sample that creates this issue, but have been unable to. The more that I observe this appears to be more related to #130 and that there is a component capability defined or introduced in the build. Also to clarify the OP stated that 5.2.0 works, but 6.0.0 breaks. I've also tried both sides of that puzzle with no errors reproduced.

This repo shows that shadow 6.0.0 as described above working successfully and not producing an error when running snyk test: https://github.com/shanman190/snyk-gradle-plugin-gh-134/tree/master (single module with shadow 6.0.0 plugin) https://github.com/shanman190/snyk-gradle-plugin-gh-134/tree/multi (multi module with shadow 6.0.0 plugin)

@mapoulin, it would help if you could provide a reproducible example that we could work from. Thanks!

[mapoulin]

@shanman190 thanks for trying to reproduce.

I've played a bit with the repo you provided. Turns out it might have to do with the fact that my project is pulling on a Java 11 dependency. Using shadow@5.2.0 I have no issue with that dependency, but when I use shadow@6.0.0 I suddenly get the error above.

Snyk seems to be looking for a version compatible with Java 8.

I've tried to give a POC here: https://github.com/shanman190/snyk-gradle-plugin-gh-134/pull/1

[shanman190]

Ahh! That makes more sense. That would mean component metadata produced by publishing would be being introduced. That then leads into your error here.

@mapoulin, just to double check, both the dependency and your project are compiling using Java 11, correct?

[shanman190]

@mapoulin, it also seems like the 6.0.0 version actually has a bug (force sets Java 1.8 because there is no easy public way to get the compiler version) that would make it partially incompatible with your use case. The Gradle team is trying to come up with a better alternative, but you can see the issue here:

https://github.com/johnrengelman/shadow/commit/dd632ba5751634aafa0c24bd0fea017368b15893#diff-996b8596add64d35b303abd9af5cb49d

I think with that information, this definitely folds back into the same issue related to component metadata and the CLI flattening all configurations into a single one for dependency analysis.

[mapoulin]

@shanman190 Yes, both of my projects are configured to use Java 11.

I'm glad this could shed some light. Thanks for your time!

[anthogez]

With this https://github.com/snyk/snyk-gradle-plugin/pull/155 we solved the issue, closing this one! ^^