search

CycloneDX / cyclonedx-maven-plugin

Creates CycloneDX Software Bill of Materials (SBOM) from Maven projects
https://cyclonedx.org/
Apache License 2.0
298 stars 86 forks source link

Snapshots being downloaded before modules are built #410

Open coheigea opened 1 year ago

coheigea commented 1 year ago

Hi,

In Apache WSS4J we have configured the plugin like:

<plugin>
                    <groupId>org.cyclonedx</groupId>
                    <artifactId>cyclonedx-maven-plugin</artifactId>
                    <version>2.7.9</version>
                    <executions>
                        <execution>
                            <id>make-bom</id>
                            <phase>package</phase>
                            <goals>
                                <goal>makeAggregateBom</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <outputName>${project.artifactId}-${project.version}-bom</outputName>
                    </configuration>
                </plugin>

https://github.com/apache/ws-wss4j/blob/17f33f0cf0680696051117098881df036bd47653/pom.xml#L333C17-L349C26

However on a normal build, it is downloading SNAPSHOTs before it even builds the modules. Also reported for Apache CXF which uses the same configuration here https://github.com/apache/cxf/pull/1480#discussion_r1365443501

nscuro commented 1 year ago

I'm not too deep into how the plugin works, but I also don't quite understand the problem here. Could you elaborate a bit more @coheigea?

IIRC, the plugin will be invoked for every module in the Maven reactor, so seeing an execution per module is expected. In order to determine the libraries used, and how they're related, the plugin needs to resolve all dependencies first. If a module depends on SNAPSHOT libraries, then those would be downloaded (or resolved from ~/.m2).

reta commented 1 year ago

IIRC, the plugin will be invoked for every module in the Maven reactor, so seeing an execution per module is expected. In order to determine the libraries used, and how they're related, the plugin needs to resolve all dependencies first. If a module depends on SNAPSHOT libraries, then those would be downloaded (or resolved from ~/.m2).

@nscuro I will try to share more details here (using https://github.com/apache/cxf/pull/1480#discussion_r1365443501) as an example. Let say we want to have a build of the Apache CXF from main branch (which currently has version4.0.4-SNAPSHOT version) but we bumping it to 4.0.5-SNAPSHOT (so we have a brand new version). The problem is that cyclonedx-maven-plugin tries to fetch 4.0.5-SNAPSHOT of the Apache CXF before they are built actually.

[INFO] -------------------------< org.apache.cxf:cxf >-------------------------
[INFO] Building Apache CXF 4.0.5-SNAPSHOT                               [1/188]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- enforcer:3.4.1:enforce (enforce-maven) @ cxf ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] Rule 2: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- cyclonedx:2.7.10:makeAggregateBom (make-bom) @ cxf ---
[INFO] CycloneDX: Resolving Aggregated Dependencies
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-core/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-core/4.0.5-SNAPSHOT/cxf-core-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-wsdl/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-wsdl/4.0.5-SNAPSHOT/cxf-rt-wsdl-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-databinding-jaxb/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-databinding-jaxb/4.0.5-SNAPSHOT/cxf-rt-databinding-jaxb-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-common/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-bindings-soap/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-common/4.0.5-SNAPSHOT/cxf-tools-common-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-bindings-soap/4.0.5-SNAPSHOT/cxf-rt-bindings-soap-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-common/4.0.5-SNAPSHOT/cxf-tools-common-4.0.5-SNAPSHOT-tests.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-validator/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-validator/4.0.5-SNAPSHOT/cxf-tools-validator-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-wsdlto-core/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-wsdlto-core/4.0.5-SNAPSHOT/cxf-tools-wsdlto-core-4.0.5-SNAPSHOT.jar
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-tools-wsdlto-frontend-jaxws/4.0.5-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots: https://repository.apache.org/snapshots/org/apache/cxf/cxf-rt-javascript/4.0.5-SNAPSHOT/maven-metadata.xml
...

To generalize the problem a bit, the plugin tries to fetch the dependencies of the modules before building these modules (fetching the external snapshots are totally fine). Hope it is helpful

ppkarwasz commented 1 year ago

I can confirm this issue. When we bind the makeAggregateBom goal to the lifecycle:

  1. The root module is the first module of the reactor,
  2. The CycloneDX plugin has therefore no way to access the project's JARs, so it downloads the previous snapshot and uses its hashes in the SBOM.

Therefore:

prabhu commented 7 months ago

@ppkarwasz are you talking about the hashes for the metadata.component (parent component) being incorrect? Have you seen issues with the components list?

ppkarwasz commented 7 months ago

@prabhu,

Looking at this again, metadata.component doesn't have any hashes and neither does any of the component elements from a Maven multi-module build. See log4j-bom-2.23.1-cyclonedx.xml for example: neither log4j-bom, nor any of the sub-modules (log4j-api, log4j-core) have a hashes key.

If there were hashes in those components, they would certainly be wrong, because the goal makeAggregateBom for log4j-bom is executed before any of the jar tasks that create the artifacts.

prabhu commented 7 months ago

@ppkarwasz Thank you. Best to track this as a separate issue, since the hashes are missed for key components. Is there an easy project to reproduce this?

garydgregory commented 6 months ago

This is a problem for Apache Log4j as well.