CycloneDX / cyclonedx-gradle-plugin

Creates CycloneDX Software Bill of Materials (SBOM) from Gradle projects
https://cyclonedx.org/
Apache License 2.0
159 stars 76 forks source link

How to configure skipping sub-projects? #77

Closed Jaff closed 3 months ago

Jaff commented 4 years ago

We have a gradle android project that pulls in a dependant sub-project, which we are scanning by default. That behavior is causing our overall scan to fail. Is there some setting that will prevent scanning the sub-project? I am using

plugins {
   id 'org.cyclonedx.bom' version '1.2.0'
}

Build output contains:

BOM inclusion for configuration debugAndroidTestApiDependenciesMetadata : []
BOM inclusion for configuration debugAndroidTestCompile : []
:app:cyclonedxBom took 103ms
Task :app:cyclonedxBom in app Finished
:app:cyclonedxBom (Thread[Execution worker for ':',5,main]) completed. Took 1.782 secs.

> Task :domain:kaptGenerateStubsTestKotlin
Task :domain:kaptGenerateStubsTestKotlin in app Starting
Deleting stale output file: /sandbox/workspace/proton-android_sonarqube/fission/domain/build/tmp/kapt3/stubs/test
Build cache key for task ':domain:kaptGenerateStubsTestKotlin' is d0234ce42e26500c3061002441e4cb05
Task ':domain:kaptGenerateStubsTestKotlin' is not up-to-date because:
  No history is available.
The input changes require a full rebuild for incremental task ':domain:kaptGenerateStubsTestKotlin'.
file or directory '/sandbox/workspace/proton-android_sonarqube/fission/domain/src/test/java', not found
file or directory '/sandbox/workspace/proton-android_sonarqube/fission/domain/src/test/java', not found
Using Kotlin/JVM incremental compilation
i: found daemon on port 17962 (278000 ms old), trying to connect
i: connected to the daemon
Options for KOTLIN DAEMON: IncrementalCompilationOptions(super=CompilationOptions(compilerMode=INCREMENTAL_COMPILER, targetPlatform=JVM, reportCategories=[0], reportSeverity=2, requestedCompilationResults=[0]kotlinScriptExtensions=[kt, kts]), areFileChangesKnown=false, modifiedFiles=null, deletedFiles=null, workingDir=/sandbox/workspace/proton-android_sonarqube/fission/domain/build/kotlin/kaptGenerateStubsTestKotlin, multiModuleICSettings=MultiModuleICSettings(buildHistoryFile=/sandbox/workspace/proton-android_sonarqube/fission/domain/build/kotlin/kaptGenerateStubsTestKotlin/build-history.bin, useModuleDetection=false), usePreciseJavaTracking=trueoutputFiles=[/sandbox/workspace/proton-android_sonarqube/fission/domain/build/tmp/kapt3/incrementalData/test, /sandbox/workspace/proton-android_sonarqube/fission/domain/build/tmp/kapt3/stubs/test, /sandbox/workspace/proton-android_sonarqube/fission/domain/build/kotlin/kaptGenerateStubsTestKotlin])
Stored cache entry for task ':domain:kaptGenerateStubsTestKotlin' with cache key d0234ce42e26500c3061002441e4cb05
:domain:kaptGenerateStubsTestKotlin took 204ms
Task :domain:kaptGenerateStubsTestKotlin in app Finished
:domain:kaptGenerateStubsTestKotlin (Thread[Execution worker for ':' Thread 6,5,main]) completed. Took 0.453 secs.

This may be more of a gradle issue, will also ask there

Jaff commented 4 years ago

Additional log detail:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:cyclonedxBom'.
> Could not resolve all dependencies for configuration ':app:debugAndroidTestCompileClasspath'.
   > More than one variant of project :data matches the consumer attributes:
       - Configuration ':data:debugApiElements' variant android-aidl:
           - Unmatched attributes:
               - Found artifactType 'android-aidl' but wasn't required.
               - Found com.android.build.api.attributes.VariantAttr 'debug' but wasn't required.
           - Compatible attributes:
               - Required com.android.build.api.attributes.BuildTypeAttr 'debug' and found compatible value 'debug'.
               - Required org.gradle.usage 'java-api' and found compatible value 'java-api'.
               - Required org.jetbrains.kotlin.platform.type 'androidJvm' and found compatible value 'androidJvm'.
       - Configuration ':data:debugApiElements' variant android-manifest:
           - Unmatched attributes:
               - Found artifactType 'android-manifest' but wasn't required.
               - Found com.android.build.api.attributes.VariantAttr 'debug' but wasn't required.
           - Compatible attributes:
               - Required com.android.build.api.attributes.BuildTypeAttr 'debug' and found compatible value 'debug'.
               - Required org.gradle.usage 'java-api' and found compatible value 'java-api'.
               - Required org.jetbrains.kotlin.platform.type 'androidJvm' and found compatible value 'androidJvm'.
       - Configuration ':data:debugApiElements' variant android-renderscript:
           - Unmatched attributes:
               - Found artifactType 'android-renderscript' but wasn't required.
               - Found com.android.build.api.attributes.VariantAttr 'debug' but wasn't required.
           - Compatible attributes:
               - Required com.android.build.api.attributes.BuildTypeAttr 'debug' and found compatible value 'debug'.
               - Required org.gradle.usage 'java-api' and found compatible value 'java-api'.
               - Required org.jetbrains.kotlin.platform.type 'androidJvm' and found compatible value 'androidJvm'.
       - Configuration ':data:debugApiElements' variant android-symbol-with-package-name:
           - Unmatched attributes:
               - Found artifactType 'android-symbol-with-package-name' but wasn't required.
               - Found com.android.build.api.attributes.VariantAttr 'debug' but wasn't required.
           - Compatible attributes:
               - Required com.android.build.api.attributes.BuildTypeAttr 'debug' and found compatible value 'debug'.
               - Required org.gradle.usage 'java-api' and found compatible value 'java-api'.
               - Required org.jetbrains.kotlin.platform.type 'androidJvm' and found compatible value 'androidJvm'.
ShreeshaDas commented 4 years ago

Hi, Same issue I am also facing and we have different build flavors also. So I don't know how the cycloneDX will work for different build flavors. Waiting for help. Thank you.

Didier116 commented 4 years ago

Hi, I am having the same, I try to bypassing it using "skipConfig" which also doesn't work. I openned on other issue about this properties issues 82. Hoping that this can unblock us.

Jaff commented 4 years ago

Mention in cyclonedx.slack.com #java-gradle channel

stevespringett commented 4 years ago

I don't use Gradle but if someone can publish a project on GitHub along with the exact commands necessary to reproduce, that would be extremely useful. You might also want to explain what the commands do along with the expected result.

llamahunter commented 4 years ago

Are you applying the plugin at the top level, or only to specific subprojects? If not applying just to the subprojects you want, maybe try that?

See: https://docs.gradle.org/current/userguide/multi_project_builds.html#sec:subproject_configuration https://docs.gradle.org/current/userguide/plugins.html#sec:subprojects_plugins_dsl https://stackoverflow.com/questions/12077083/what-is-the-difference-between-allprojects-and-subprojects

Didier116 commented 4 years ago

I try using cyclonedx only on my subproject "app" (build.gradle) :

plugins {
    id 'org.cyclonedx.bom' apply false
}
project(':app') {
        apply plugin: 'org.cyclonedx.bom'
        cyclonedxBom {
            // skipConfigs is a list of configuration names to exclude when generating the BOM
            skipConfigs += ["compileClasspath", "testCompileClasspath","demoDebugAndroidTestCompileClasspath","app:demoDebugAndroidTestCompileClasspath",":app:demoDebugAndroidTestCompileClasspath"]
        }
}

Same error : Could not resolve all dependencies for configuration ':app:demoDebugAndroidTestRuntimeClasspath'. More than one variant of project :library matches the consumer attributes:

I think is a configuration issue due to the use of several BuildVariant (ProductFlavor/BuildType).

ibnamik commented 3 years ago

Any updates or workarounds? This is a huge blocker for me.

SamTobias commented 3 years ago

I'm was having a similar problem and "fixed" adding configuration: 'default' at the dependency:

implementation project(path: ':sub-project-a', configuration: 'default')

This makes CycloneDX works, but adding this makes the app crash with a ClassNotFoundException at runtime when calling a method from a dependency of this sub-project

vineetkala11 commented 3 years ago

Same issue for me -

* What went wrong:
Execution failed for task ':cyclonedxBom'.
> Could not resolve all dependencies for configuration ':contractTestRuntimeOnlyDependenciesMetadata'.
   > Could not find com.soprabanking.dxp:commons-monitor:.
     Required by:
         project :
   > Could not find org.junit.jupiter:junit-jupiter-engine:.
     Required by:
         project :

Fix: Skipping configuration inside build.gradle.kts, worked for me -

tasks.withType<org.cyclonedx.gradle.CycloneDxTask>().configureEach { setSkipConfigs(listOf("contractTestRuntimeOnlyDependenciesMetadata")) }

gitWK86 commented 2 years ago

I'm was having a similar problem on gradle 7.1.1 .No problem after replacing with gradle version 4.2.2 issues 109

Execution failed for task ':app:cyclonedxBom'.
> Could not resolve all dependencies for configuration ':app:debugAndroidTestCompileClasspath'.
   > The consumer was configured to find an API of a component, preferably optimized for Android, as well as attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'debug', attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.1', attribute 'org.jetbrains.kotlin.platform.type' with value 'androidJvm'. However we cannot choose between the following variants of project :app:
       - Configuration ':app:debugApiElements' variant android-base-module-metadata declares an API of a component, as well as attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.1', attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'debug', attribute 'org.jetbrains.kotlin.platform.type' with value 'androidJvm':
           - Unmatched attributes:
               - Provides attribute 'artifactType' with value 'android-base-module-metadata' but the consumer didn't ask for it
               - Provides attribute 'com.android.build.gradle.internal.attributes.VariantAttr' with value 'debug' but the consumer didn't ask for it
               - Doesn't say anything about its target Java environment (preferred optimized for Android)
       - Configuration ':app:debugApiElements' variant android-feature-all-metadata declares an API of a component, as well as attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.1', attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'debug', attribute 'org.jetbrains.kotlin.platform.type' with value 'androidJvm':
           - Unmatched attributes:
               - Provides attribute 'artifactType' with value 'android-feature-all-metadata' but the consumer didn't ask for it
               - Provides attribute 'com.android.build.gradle.internal.attributes.VariantAttr' with value 'debug' but the consumer didn't ask for it
               - Doesn't say anything about its target Java environment (preferred optimized for Android)
       - Configuration ':app:debugApiElements' variant android-feature-res-ap_ declares an API of a component, as well as attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.1', attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'debug', attribute 'org.jetbrains.kotlin.platform.type' with value 'androidJvm':
           - Unmatched attributes:
               - Provides attribute 'artifactType' with value 'android-feature-res-ap_' but the consumer didn't ask for it
               - Provides attribute 'com.android.build.gradle.internal.attributes.VariantAttr' with value 'debug' but the consumer didn't ask for it
               - Doesn't say anything about its target Java environment (preferred optimized for Android)
       - Configuration ':app:debugApiElements' variant android-feature-signing-config-data declares an API of a component, as well as attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.1', attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'debug', attribute 'org.jetbrains.kotlin.platform.type' with value 'androidJvm':
           - Unmatched attributes:
               - Provides attribute 'artifactType' with value 'android-feature-signing-config-data' but the consumer didn't ask for it
               - Provides attribute 'com.android.build.gradle.internal.attributes.VariantAttr' with value 'debug' but the consumer didn't ask for it
               - Doesn't say anything about its target Java environment (preferred optimized for Android)
       - Configuration ':app:debugApiElements' variant android-feature-signing-config-versions declares an API of a component, as well as attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.1', attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'debug', attribute 'org.jetbrains.kotlin.platform.type' with value 'androidJvm':
           - Unmatched attributes:
               - Provides attribute 'artifactType' with value 'android-feature-signing-config-versions' but the consumer didn't ask for it
               - Provides attribute 'com.android.build.gradle.internal.attributes.VariantAttr' with value 'debug' but the consumer didn't ask for it
               - Doesn't say anything about its target Java environment (preferred optimized for Android)
       - Configuration ':app:debugApiElements' variant android-java-res declares an API of a component, as well as attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.1', attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'debug', attribute 'org.jetbrains.kotlin.platform.type' with value 'androidJvm':
           - Unmatched attributes:
               - Provides attribute 'artifactType' with value 'android-java-res' but the consumer didn't ask for it
               - Provides attribute 'com.android.build.gradle.internal.attributes.VariantAttr' with value 'debug' but the consumer didn't ask for it
               - Doesn't say anything about its target Java environment (preferred optimized for Android)
       - Configuration ':app:debugApiElements' variant android-manifest-metadata declares an API of a component, as well as attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.1', attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'debug', attribute 'org.jetbrains.kotlin.platform.type' with value 'androidJvm':
           - Unmatched attributes:
               - Provides attribute 'artifactType' with value 'android-manifest-metadata' but the consumer didn't ask for it
               - Provides attribute 'com.android.build.gradle.internal.attributes.VariantAttr' with value 'debug' but the consumer didn't ask for it
               - Doesn't say anything about its target Java environment (preferred optimized for Android)

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
adam-siklosi commented 1 year ago

Hi @stevespringett

As far as I see currently you have two options to use the plugin:

1.) Apply it on the root project --> Cannot skip subprojects 2.) Apply it on subprojects separately --> Produces a bom file per subproject. You need to combine those files manually later using cyclonedx-cli merge -> so why use the plugin at all

I believe the makeAggregateBom functionality with the exclude options of the https://github.com/CycloneDX/cyclonedx-maven-plugin is missing here in the gradle plugin.

Though I'm not sure if you can easily merge the reports when the plugin is applied per subproject. So perhaps the best approach would be to apply the plugin on the root level and provide a configuration option skipProjects to filter the subprojects (analog to the skipConfigs).

https://github.com/CycloneDX/cyclonedx-gradle-plugin/blob/master/src/main/java/org/cyclonedx/gradle/CycloneDxTask.java#L241-L249

What do you think?

Thanks, Adam

skhokhlov commented 3 months ago

Implemented in https://github.com/CycloneDX/cyclonedx-gradle-plugin/pull/236

github-actions[bot] commented 2 months ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.