Merging SBOMs doesn’t get rid of duplicates

[yaourabi]

I have a Gradle multi-project build, in which i generate an SBOM for java dependencies for each project using the CycloneDX Gradle plugin. I want to merge all the existing SBOMs, however the resulting SBOM doesn’t get rid of duplicated dependencies.

Steps to reproduce

• Create a build with two sub-projects.

dependencies for the first project

dependencies {
    implementation("org.apache.commons:commons-lang3:3.12.0")
}

dependencies for the second project

dependencies {
    implementation("org.apache.commons:commons-lang3:3.12.0")
    implementation("com.fasterxml.jackson.core:jackson-databind:2.15.3")
}

• Apply the CycloneDX Gradle plugin for each project

tasks.cyclonedxBom {
    setProjectType("application")
    setSchemaVersion("1.5")
    setDestination(project.file("build/reports"))
    setOutputName("bom2")
    setOutputFormat("json")
    setIncludeBomSerialNumber(false)
    setIncludeLicenseText(true)
    setComponentVersion("2.0.0")
}

• Generate an SBOM for each project
• Use the cyclonedx-cli merge command to merge the two SBOMs

CycloneDX cli version: 0.25.0 0.25.0

CycloneDX Gradle plugin version: 1.7.4

Expected behavior

although the commons-lang3 dependency is defined in both projects, I only want it to be declared in the final SBOM once.

Current behavior

the current SBOM declares the commons-lang3 dependency twice.

"dependencies": [
    {
      "ref": "pkg:maven/org.apache.commons/commons-lang3@3.12.0?type=jar",
      "dependsOn": []
    },
    {
      "ref": "pkg:maven/org.example/app1@1.0-SNAPSHOT?type=jar",
      "dependsOn": [
        "pkg:maven/org.apache.commons/commons-lang3@3.12.0?type=jar"
      ]
    },
    {
      "ref": "pkg:maven/com.fasterxml.jackson.core/jackson-annotations@2.15.3?type=jar",
      "dependsOn": []
    },
    {
      "ref": "pkg:maven/org.apache.commons/commons-lang3@3.12.0?type=jar",
      "dependsOn": []
    },
    {
      "ref": "pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.15.3?type=jar",
      "dependsOn": [
        "pkg:maven/com.fasterxml.jackson.core/jackson-annotations@2.15.3?type=jar",
        "pkg:maven/com.fasterxml.jackson.core/jackson-core@2.15.3?type=jar"
      ]
    },
    {
      "ref": "pkg:maven/org.example/app2@1.0-SNAPSHOT?type=jar",
      "dependsOn": [
        "pkg:maven/org.apache.commons/commons-lang3@3.12.0?type=jar",
        "pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.15.3?type=jar"
      ]
    },
    {
      "ref": "pkg:maven/com.fasterxml.jackson.core/jackson-core@2.15.3?type=jar",
      "dependsOn": []
    }
  ]

[ertl]

+1

[zabulus]

+1. This blows up count of both components and vulnerabilities in dependency-track. Use-case is following:

1. We create sbom file for each .csproj separately using the cyclonedx dotnet tool.
2. Then we merge everything in single file using the cyclonedx cli tool.
3. Import resulting file to dependency-track, see over 10k components with lots of duplicates in project

[ferben]

+1 Some --deduplicate option would be very useful 👍