search

CycloneDX / cyclonedx-maven-plugin

Creates CycloneDX Software Bill of Materials (SBOM) from Maven projects
https://cyclonedx.org/
Apache License 2.0
298 stars 85 forks source link

Performance degradation in 2.7.6+ #344

Closed rovarga closed 1 year ago

rovarga commented 1 year ago

The makeBom execution suffers a massive performance degradation since 2.7.6, also present in 2.7.7:

With 2.7.5:

[INFO] Scanning for projects...
[INFO] 
[INFO] ----------< org.opendaylight.aaa:odl-aaa-encryption-service >-----------
[INFO] Building ODL :: aaa :: odl-aaa-encryption-service 0.17.8-SNAPSHOT
[INFO]   from pom.xml
[INFO] ------------------------------[ feature ]-------------------------------
[INFO] 
[INFO] --- cyclonedx:2.7.5:makeBom (default-cli) @ odl-aaa-encryption-service ---
[INFO] CycloneDX: Parameters
[INFO] ------------------------------------------------------------------------
[INFO] schemaVersion          : 1.4
[INFO] includeBomSerialNumber : true
[INFO] includeCompileScope    : true
[INFO] includeProvidedScope   : true
[INFO] includeRuntimeScope    : true
[INFO] includeTestScope       : false
[INFO] includeSystemScope     : true
[INFO] includeLicenseText     : false
[INFO] outputFormat           : all
[INFO] outputName             : bom
[INFO] ------------------------------------------------------------------------
[INFO] CycloneDX: Resolving Dependencies
[INFO] CycloneDX: Creating BOM
[INFO] CycloneDX: Writing BOM (XML): /home/nite/odl/aaa/features/odl-aaa-encryption-service/target/bom.xml
[INFO] CycloneDX: Validating BOM (XML): /home/nite/odl/aaa/features/odl-aaa-encryption-service/target/bom.xml
[INFO] CycloneDX: Writing BOM (JSON): /home/nite/odl/aaa/features/odl-aaa-encryption-service/target/bom.json
[INFO] CycloneDX: Validating BOM (JSON): /home/nite/odl/aaa/features/odl-aaa-encryption-service/target/bom.json
[WARNING] Unknown keyword additionalItems - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  8.267 s
[INFO] Finished at: 2023-04-20T08:23:41+02:00
[INFO] ------------------------------------------------------------------------

With 2.7.7:

[INFO] Scanning for projects...
[INFO] 
[INFO] ----------< org.opendaylight.aaa:odl-aaa-encryption-service >-----------
[INFO] Building ODL :: aaa :: odl-aaa-encryption-service 0.17.8-SNAPSHOT
[INFO]   from pom.xml
[INFO] ------------------------------[ feature ]-------------------------------
[INFO] 
[INFO] --- cyclonedx:2.7.7:makeBom (default-cli) @ odl-aaa-encryption-service ---
[INFO] CycloneDX: Resolving Dependencies
[INFO] CycloneDX: Creating BOM version 1.4 with 191 component(s)
[INFO] CycloneDX: Writing and validating BOM (XML): /home/nite/odl/aaa/features/odl-aaa-encryption-service/target/bom.xml
[INFO]            attaching as odl-aaa-encryption-service-0.17.8-SNAPSHOT-cyclonedx.xml
[INFO] CycloneDX: Writing and validating BOM (JSON): /home/nite/odl/aaa/features/odl-aaa-encryption-service/target/bom.json
[WARNING] Unknown keyword additionalItems - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
[INFO]            attaching as odl-aaa-encryption-service-0.17.8-SNAPSHOT-cyclonedx.json
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  36.081 s
[INFO] Finished at: 2023-04-20T08:25:04+02:00
[INFO] ------------------------------------------------------------------------

The crux of the problem seems to lie in DefaultModelConverter.generatePackageUrl() and DefaultProjectDependenciesConverter.buildDependencyGraphNode() -- these get called ~22M and ~7M times respectively. cyclonedx-277-hotspots

rovarga commented 1 year ago

The majority of the time spent seems to be in PackageURL.canonicalize(). In 2.7.5 there were 12,214 invocations accounting for 31ms (plus 20ms for constructor) . In 2.7.7 there are 21,996,184 invocations accounting for 35,273ms (plus 31,145ms for constructor). cyclonedx-277-gpu

rovarga commented 1 year ago

FYI, this is what the project dependency tree looks like:

[INFO] org.opendaylight.aaa:odl-aaa-encryption-service:feature:0.17.8-SNAPSHOT
[INFO] +- org.opendaylight.controller:odl-mdsal-broker:xml:features:7.0.5:compile
[INFO] |  +- org.opendaylight.mdsal:odl-mdsal-eos-binding:xml:features:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:odl-mdsal-eos-dom:xml:features:11.0.8:compile
[INFO] |  |  |  +- org.opendaylight.mdsal:odl-mdsal-eos-common:xml:features:11.0.8:compile
[INFO] |  |  |  |  \- org.opendaylight.mdsal:mdsal-eos-common-spi:jar:11.0.8:compile
[INFO] |  |  |  \- org.opendaylight.mdsal:mdsal-eos-dom-api:jar:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:odl-mdsal-binding-dom-adapter:xml:features:11.0.8:compile
[INFO] |  |  |  \- org.opendaylight.mdsal:mdsal-binding-dom-adapter:jar:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:mdsal-eos-binding-api:jar:11.0.8:compile
[INFO] |  |  |  +- org.opendaylight.mdsal:mdsal-eos-common-api:jar:11.0.8:compile
[INFO] |  |  |  \- org.opendaylight.mdsal.model:general-entity:jar:11.0.8:compile
[INFO] |  |  \- org.opendaylight.mdsal:mdsal-eos-binding-adapter:jar:11.0.8:compile
[INFO] |  |     \- org.opendaylight.mdsal:mdsal-binding-dom-codec-spi:jar:11.0.8:compile
[INFO] |  +- org.opendaylight.mdsal:odl-mdsal-singleton-dom:xml:features:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:odl-mdsal-singleton-common:xml:features:11.0.8:compile
[INFO] |  |  |  \- org.opendaylight.mdsal:mdsal-singleton-common-api:jar:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:odl-mdsal-common:xml:features:11.0.8:compile
[INFO] |  |  |  \- org.opendaylight.yangtools:odl-yangtools-common:xml:features:10.0.5:compile
[INFO] |  |  |     \- org.opendaylight.yangtools:odl-yangtools-util:xml:features:10.0.5:compile
[INFO] |  |  |        +- tech.pantheon.triemap:pt-triemap:xml:features:1.2.0:compile
[INFO] |  |  |        \- org.opendaylight.odlparent:odl-guava:xml:features:12.0.4:compile
[INFO] |  |  \- org.opendaylight.mdsal:mdsal-singleton-dom-impl:jar:11.0.8:compile
[INFO] |  +- org.opendaylight.controller:odl-mdsal-remoterpc-connector:xml:features:7.0.5:compile
[INFO] |  |  \- org.opendaylight.controller:sal-remoterpc-connector:jar:7.0.5:compile
[INFO] |  |     +- org.opendaylight.controller:repackaged-akka:jar:7.0.5:compile
[INFO] |  |     |  +- com.typesafe:config:jar:1.4.2:compile
[INFO] |  |     |  +- com.typesafe:ssl-config-core_2.13:jar:0.4.3:compile
[INFO] |  |     |  +- io.aeron:aeron-client:jar:1.38.1:compile
[INFO] |  |     |  +- io.aeron:aeron-driver:jar:1.38.1:compile
[INFO] |  |     |  +- io.netty:netty:jar:3.10.6.Final:compile
[INFO] |  |     |  +- org.agrona:agrona:jar:1.15.2:compile
[INFO] |  |     |  +- org.reactivestreams:reactive-streams:jar:1.0.4:compile
[INFO] |  |     |  +- org.lmdbjava:lmdbjava:jar:0.7.0:compile
[INFO] |  |     |  +- org.scala-lang:scala-reflect:jar:2.13.10:compile
[INFO] |  |     |  \- org.scala-lang.modules:scala-parser-combinators_2.13:jar:1.1.2:compile
[INFO] |  |     +- org.opendaylight.controller:sal-common-util:jar:7.0.5:compile
[INFO] |  |     +- org.opendaylight.controller:sal-clustering-commons:jar:7.0.5:compile
[INFO] |  |     |  +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] |  |     |  +- io.dropwizard.metrics:metrics-core:jar:4.2.18:compile
[INFO] |  |     |  +- io.dropwizard.metrics:metrics-graphite:jar:4.2.18:compile
[INFO] |  |     |  |  \- com.rabbitmq:amqp-client:jar:5.16.0:compile
[INFO] |  |     |  +- io.dropwizard.metrics:metrics-jmx:jar:4.2.18:compile
[INFO] |  |     |  \- org.lz4:lz4-java:jar:1.8.0:compile
[INFO] |  |     \- org.scala-lang:scala-library:jar:2.13.10:compile
[INFO] |  \- org.opendaylight.controller:odl-mdsal-distributed-datastore:xml:features:7.0.5:compile
[INFO] |     +- org.opendaylight.odlparent:odl-apache-commons-text:xml:features:12.0.4:compile
[INFO] |     |  +- org.opendaylight.odlparent:odl-apache-commons-lang3:xml:features:12.0.4:compile
[INFO] |     |  \- org.apache.commons:commons-text:jar:1.10.0:compile
[INFO] |     +- org.opendaylight.mdsal:odl-mdsal-dom-broker:xml:features:11.0.8:compile
[INFO] |     |  \- org.opendaylight.mdsal:mdsal-dom-broker:jar:11.0.8:compile
[INFO] |     |     \- org.opendaylight.yangtools:yang-parser-api:jar:10.0.5:compile
[INFO] |     +- org.opendaylight.controller:odl-mdsal-clustering-commons:xml:features:7.0.5:compile
[INFO] |     |  +- org.opendaylight.controller:odl-controller-akka:xml:features:7.0.5:compile
[INFO] |     |  |  \- org.opendaylight.controller:odl-controller-scala:xml:features:7.0.5:compile
[INFO] |     |  +- org.opendaylight.odlparent:odl-dropwizard-metrics:xml:features:12.0.4:compile
[INFO] |     |  |  +- io.dropwizard.metrics:metrics-healthchecks:jar:4.2.18:compile
[INFO] |     |  |  \- io.dropwizard.metrics:metrics-jvm:jar:4.2.18:compile
[INFO] |     |  +- org.opendaylight.odlparent:odl-servlet-api:xml:features:12.0.4:compile
[INFO] |     |  +- org.opendaylight.controller:sal-akka-raft:jar:7.0.5:compile
[INFO] |     |  \- org.opendaylight.controller:sal-akka-segmented-journal:jar:7.0.5:compile
[INFO] |     |     \- org.opendaylight.controller:atomix-storage:jar:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:odl-controller-mdsal-common:xml:features:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:cds-access-api:jar:7.0.5:compile
[INFO] |     |  \- org.opendaylight.yangtools:yang-data-tree-api:jar:10.0.5:compile
[INFO] |     +- org.opendaylight.controller:cds-access-client:jar:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:cds-dom-api:jar:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:sal-distributed-datastore:jar:7.0.5:compile
[INFO] |     |  +- org.scala-lang.modules:scala-java8-compat_2.13:jar:1.0.2:compile
[INFO] |     |  +- net.java.dev.stax-utils:stax-utils:jar:20070216:compile
[INFO] |     |  +- org.opendaylight.controller:cds-mgmt-api:jar:7.0.5:compile
[INFO] |     |  +- org.opendaylight.yangtools:yang-data-tree-spi:jar:10.0.5:compile
[INFO] |     |  +- org.opendaylight.yangtools:yang-data-tree-ri:jar:10.0.5:compile
[INFO] |     |  +- org.opendaylight.yangtools:yang-data-util:jar:10.0.5:compile
[INFO] |     |  \- org.opendaylight.yangtools:yang-model-spi:jar:10.0.5:compile
[INFO] |     |     \- org.opendaylight.yangtools:rfc7952-model-api:jar:10.0.5:compile
[INFO] |     +- org.opendaylight.controller:eos-dom-akka:jar:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:sal-cluster-admin-api:jar:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:sal-cluster-admin-impl:jar:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:sal-clustering-config:xml:akkaconf:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:sal-clustering-config:xml:factoryakkaconf:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:sal-clustering-config:xml:moduleshardconf:7.0.5:compile
[INFO] |     +- org.opendaylight.controller:sal-clustering-config:xml:moduleconf:7.0.5:compile
[INFO] |     \- org.opendaylight.controller:sal-clustering-config:cfg:datastore:7.0.5:compile
[INFO] +- org.opendaylight.controller:odl-controller-blueprint:xml:features:7.0.5:compile
[INFO] |  +- org.opendaylight.yangtools:odl-yangtools-codec:xml:features:10.0.5:compile
[INFO] |  |  +- org.opendaylight.odlparent:odl-gson:xml:features:12.0.4:compile
[INFO] |  |  |  \- com.google.code.gson:gson:jar:2.10.1:compile
[INFO] |  |  +- org.opendaylight.odlparent:odl-stax2-api:xml:features:12.0.4:compile
[INFO] |  |  |  \- org.codehaus.woodstox:stax2-api:jar:4.2.1:compile
[INFO] |  |  +- org.opendaylight.yangtools:odl-yangtools-data:xml:features:10.0.5:compile
[INFO] |  |  |  +- org.opendaylight.odlparent:odl-antlr4:xml:features:12.0.4:compile
[INFO] |  |  |  |  \- org.antlr:antlr4-runtime:jar:4.12.0:compile
[INFO] |  |  |  \- org.opendaylight.yangtools:yang-data-transform:jar:10.0.5:compile
[INFO] |  |  +- org.opendaylight.yangtools:yang-data-codec-binfmt:jar:10.0.5:compile
[INFO] |  |  |  \- org.opendaylight.yangtools:rfc8528-data-api:jar:10.0.5:compile
[INFO] |  |  +- org.opendaylight.yangtools:yang-data-codec-gson:jar:10.0.5:compile
[INFO] |  |  \- org.opendaylight.yangtools:yang-data-codec-xml:jar:10.0.5:compile
[INFO] |  |     \- org.opendaylight.yangtools:rfc7952-data-api:jar:10.0.5:compile
[INFO] |  +- org.opendaylight.mdsal:odl-mdsal-binding-api:xml:features:11.0.8:compile
[INFO] |  |  +- org.opendaylight.yangtools:odl-yangtools-data-api:xml:features:10.0.5:compile
[INFO] |  |  |  +- org.opendaylight.yangtools:odl-yangtools-parser-api:xml:features:10.0.5:compile
[INFO] |  |  |  |  +- org.opendaylight.yangtools:odl-yangtools-xpath-api:xml:features:10.0.5:compile
[INFO] |  |  |  |  +- org.opendaylight.yangtools:openconfig-model-api:jar:10.0.5:compile
[INFO] |  |  |  |  +- org.opendaylight.yangtools:rfc6241-model-api:jar:10.0.5:compile
[INFO] |  |  |  |  +- org.opendaylight.yangtools:rfc6536-model-api:jar:10.0.5:compile
[INFO] |  |  |  |  +- org.opendaylight.yangtools:rfc6643-model-api:jar:10.0.5:compile
[INFO] |  |  |  |  +- org.opendaylight.yangtools:rfc8528-model-api:jar:10.0.5:compile
[INFO] |  |  |  |  +- org.opendaylight.yangtools:rfc8639-model-api:jar:10.0.5:compile
[INFO] |  |  |  |  \- org.opendaylight.yangtools:rfc8819-model-api:jar:10.0.5:compile
[INFO] |  |  |  +- org.opendaylight.yangtools:yang-data-spi:jar:10.0.5:compile
[INFO] |  |  |  +- org.opendaylight.yangtools:rfc7952-data-util:jar:10.0.5:compile
[INFO] |  |  |  \- org.opendaylight.yangtools:rfc8528-data-util:jar:10.0.5:compile
[INFO] |  |  +- org.opendaylight.mdsal:odl-mdsal-binding-base:xml:features:11.0.8:compile
[INFO] |  |  |  \- org.opendaylight.mdsal.model:yang-ext:jar:2013.09.07.23.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:mdsal-binding-spi:jar:11.0.8:compile
[INFO] |  |  \- org.opendaylight.mdsal:mdsal-binding-util:jar:11.0.8:compile
[INFO] |  +- org.opendaylight.mdsal:odl-mdsal-binding-runtime:xml:features:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:odl-mdsal-bytebuddy:xml:features:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:odl-mdsal-dom-runtime:xml:features:11.0.8:compile
[INFO] |  |  |  +- org.opendaylight.mdsal:odl-mdsal-binding-runtime-api:xml:features:11.0.8:compile
[INFO] |  |  |  |  \- org.opendaylight.yangtools:odl-yangtools-parser:xml:features:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:odl-yangtools-xpath:xml:features:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:odlext-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:openconfig-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:rfc6241-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:rfc6536-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:rfc6643-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:rfc7952-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:rfc8040-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:rfc8528-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:rfc8639-parser-support:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:yang-parser-reactor:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:yang-parser-rfc7950:jar:10.0.5:compile
[INFO] |  |  |  |     +- org.opendaylight.yangtools:yang-parser-spi:jar:10.0.5:compile
[INFO] |  |  |  |     \- org.opendaylight.yangtools:yang-repo-fs:jar:10.0.5:compile
[INFO] |  |  |  \- org.opendaylight.mdsal:mdsal-dom-schema-osgi:jar:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:mdsal-binding-generator:jar:11.0.8:compile
[INFO] |  |  |  +- org.opendaylight.yangtools:odlext-model-api:jar:10.0.5:compile
[INFO] |  |  |  +- org.opendaylight.yangtools:rfc8040-model-api:jar:10.0.5:compile
[INFO] |  |  |  +- org.opendaylight.yangtools:yang-model-ri:jar:10.0.5:compile
[INFO] |  |  |  +- org.opendaylight.yangtools:yang-xpath-api:jar:10.0.5:compile
[INFO] |  |  |  +- org.opendaylight.mdsal:mdsal-binding-model-api:jar:11.0.8:compile
[INFO] |  |  |  +- org.opendaylight.mdsal:mdsal-binding-model-ri:jar:11.0.8:compile
[INFO] |  |  |  \- org.opendaylight.mdsal:mdsal-binding-runtime-api:jar:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:mdsal-binding-dom-codec:jar:11.0.8:compile
[INFO] |  |  |  \- org.opendaylight.mdsal:mdsal-binding-loader:jar:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:mdsal-binding-dom-codec-osgi:jar:11.0.8:compile
[INFO] |  |  \- org.opendaylight.mdsal:mdsal-binding-runtime-osgi:jar:11.0.8:compile
[INFO] |  |     \- org.opendaylight.mdsal:mdsal-binding-runtime-spi:jar:11.0.8:compile
[INFO] |  |        \- org.opendaylight.yangtools:yang-parser-impl:jar:10.0.5:compile
[INFO] |  |           +- org.opendaylight.yangtools:rfc8819-parser-support:jar:10.0.5:compile
[INFO] |  |           \- org.opendaylight.yangtools:yang-xpath-impl:jar:10.0.5:compile
[INFO] |  +- org.opendaylight.mdsal:odl-mdsal-dom-api:xml:features:11.0.8:compile
[INFO] |  |  +- org.opendaylight.mdsal:mdsal-dom-api:jar:11.0.8:compile
[INFO] |  |  |  +- org.opendaylight.yangtools:yang-repo-api:jar:10.0.5:compile
[INFO] |  |  |  |  \- org.opendaylight.yangtools:yang-ir:jar:10.0.5:compile
[INFO] |  |  |  \- org.opendaylight.yangtools:yang-repo-spi:jar:10.0.5:compile
[INFO] |  |  \- org.opendaylight.mdsal:mdsal-dom-spi:jar:11.0.8:compile
[INFO] |  \- org.opendaylight.controller:blueprint:jar:7.0.5:compile
[INFO] |     +- org.apache.aries:org.apache.aries.util:jar:1.1.3:compile
[INFO] |     +- org.opendaylight.yangtools:concepts:jar:10.0.5:compile
[INFO] |     +- org.opendaylight.yangtools:util:jar:10.0.5:compile
[INFO] |     |  \- tech.pantheon.triemap:triemap:jar:1.2.0:compile
[INFO] |     +- org.opendaylight.yangtools:yang-data-api:jar:10.0.5:compile
[INFO] |     +- org.opendaylight.yangtools:yang-data-impl:jar:10.0.5:compile
[INFO] |     +- org.opendaylight.yangtools:yang-model-api:jar:10.0.5:compile
[INFO] |     +- org.opendaylight.yangtools:yang-model-util:jar:10.0.5:compile
[INFO] |     +- org.opendaylight.mdsal:mdsal-binding-dom-codec-api:jar:11.0.8:compile
[INFO] |     +- org.opendaylight.mdsal:mdsal-binding-spec-util:jar:11.0.8:compile
[INFO] |     \- org.opendaylight.mdsal:yang-binding:jar:11.0.8:compile
[INFO] +- org.opendaylight.aaa:aaa-encrypt-service:jar:0.17.8-SNAPSHOT:compile
[INFO] |  +- org.bouncycastle:bcpkix-jdk18on:jar:1.72:compile
[INFO] |  |  \- org.bouncycastle:bcutil-jdk18on:jar:1.72:compile
[INFO] |  \- org.bouncycastle:bcprov-jdk18on:jar:1.72:compile
[INFO] +- org.opendaylight.aaa:aaa-encrypt-service-impl:jar:0.17.8-SNAPSHOT:compile
[INFO] |  +- com.google.guava:guava:jar:31.1-jre:compile
[INFO] |  |  +- com.google.guava:failureaccess:jar:1.0.1:compile
[INFO] |  |  +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO] |  |  +- org.checkerframework:checker-qual:jar:3.33.0:compile
[INFO] |  |  +- com.google.errorprone:error_prone_annotations:jar:2.18.0:compile
[INFO] |  |  \- com.google.j2objc:j2objc-annotations:jar:1.3:compile
[INFO] |  +- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] |  +- org.opendaylight.odlparent:logging-markers:jar:12.0.4:compile
[INFO] |  +- org.opendaylight.yangtools:yang-common:jar:10.0.5:compile
[INFO] |  +- org.opendaylight.mdsal:mdsal-binding-api:jar:11.0.8:compile
[INFO] |  \- org.opendaylight.mdsal:mdsal-common-api:jar:11.0.8:compile
[INFO] +- org.opendaylight.odlparent:features-test:jar:12.0.4:test
[INFO] |  +- org.ops4j.pax.exam:pax-exam-container-karaf:jar:4.13.5:test
[INFO] |  |  +- org.ops4j.pax.exam:pax-exam-container-remote:jar:4.13.5:test
[INFO] |  |  |  +- org.ops4j.pax.exam:pax-exam-container-rbc-client:jar:4.13.5:test
[INFO] |  |  |  |  \- org.ops4j.pax.exam:pax-exam-container-rbc:jar:4.13.5:test
[INFO] |  |  |  \- org.ops4j.pax.swissbox:pax-swissbox-core:jar:1.8.4:test
[INFO] |  |  |     \- org.ops4j.pax.swissbox:pax-swissbox-lifecycle:jar:1.8.4:test
[INFO] |  |  +- org.apache.commons:commons-compress:jar:1.22:test
[INFO] |  |  +- org.ops4j.pax.swissbox:pax-swissbox-framework:jar:1.8.4:test
[INFO] |  |  |  +- org.ops4j.pax.swissbox:pax-swissbox-tracker:jar:1.8.4:test
[INFO] |  |  |  \- org.ops4j.base:ops4j-base-monitors:jar:1.5.1:test
[INFO] |  |  +- org.ops4j.base:ops4j-base-spi:jar:1.5.1:test
[INFO] |  |  +- commons-io:commons-io:jar:2.11.0:test
[INFO] |  |  \- org.apache.felix:org.apache.felix.configadmin:jar:1.9.26:test
[INFO] |  +- org.ops4j.pax.exam:pax-exam-junit4:jar:4.13.5:test
[INFO] |  |  +- org.ops4j.pax.exam:pax-exam-spi:jar:4.13.5:test
[INFO] |  |  |  \- org.ops4j.pax.tinybundles:tinybundles:jar:3.0.0:test
[INFO] |  |  \- org.ops4j.base:ops4j-base-lang:jar:1.5.1:test
[INFO] |  +- org.ops4j.pax.exam:pax-exam:jar:4.13.5:test
[INFO] |  |  +- org.ops4j.base:ops4j-base-exec:jar:1.5.1:test
[INFO] |  |  +- org.ops4j.base:ops4j-base-io:jar:1.5.1:test
[INFO] |  |  +- org.ops4j.base:ops4j-base-store:jar:1.5.1:test
[INFO] |  |  \- org.ops4j.base:ops4j-base-util-property:jar:1.5.1:test
[INFO] |  +- org.ops4j.pax.url:pax-url-aether:jar:2.6.12:test
[INFO] |  |  +- org.ops4j.pax.url:pax-url-aether-support:jar:2.6.12:test
[INFO] |  |  |  \- org.apache.maven.resolver:maven-resolver-impl:jar:1.8.2:test
[INFO] |  |  |     +- org.apache.maven.resolver:maven-resolver-api:jar:1.8.2:test
[INFO] |  |  |     +- org.apache.maven.resolver:maven-resolver-spi:jar:1.8.2:test
[INFO] |  |  |     +- org.apache.maven.resolver:maven-resolver-named-locks:jar:1.8.2:test
[INFO] |  |  |     +- org.apache.maven.resolver:maven-resolver-util:jar:1.8.2:test
[INFO] |  |  |     \- javax.annotation:javax.annotation-api:jar:1.3.2:test
[INFO] |  |  \- org.slf4j:jcl-over-slf4j:jar:1.7.32:test
[INFO] |  +- org.apache.karaf.features:standard:xml:features:4.4.3:test
[INFO] |  +- org.apache.karaf.deployer:org.apache.karaf.deployer.blueprint:jar:4.4.3:test
[INFO] |  +- org.apache.karaf.deployer:org.apache.karaf.deployer.features:jar:4.4.3:test
[INFO] |  |  \- org.apache.karaf.features:org.apache.karaf.features.core:jar:4.4.3:test
[INFO] |  |     +- org.ops4j.base:ops4j-base-util-collections:jar:1.5.1:test
[INFO] |  |     +- org.ops4j.pax.swissbox:pax-swissbox-property:jar:1.8.5:test
[INFO] |  |     +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.3:test
[INFO] |  |     |  \- jakarta.activation:jakarta.activation-api:jar:1.2.2:test
[INFO] |  |     +- org.glassfish.jaxb:jaxb-runtime:jar:2.3.3:test
[INFO] |  |     |  +- org.glassfish.jaxb:txw2:jar:2.3.3:test
[INFO] |  |     |  +- com.sun.istack:istack-commons-runtime:jar:3.0.10:test
[INFO] |  |     |  \- com.sun.activation:jakarta.activation:jar:1.2.2:test
[INFO] |  |     +- com.fasterxml.jackson.core:jackson-databind:jar:2.14.2:test
[INFO] |  |     |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.14.2:test
[INFO] |  |     |  \- com.fasterxml.jackson.core:jackson-core:jar:2.14.2:test
[INFO] |  |     \- javax.activation:activation:jar:1.1.1:test
[INFO] |  +- org.ops4j.pax.url:pax-url-wrap:jar:2.6.12:test
[INFO] |  |  +- org.ops4j.pax.url:pax-url-commons:jar:2.6.12:test
[INFO] |  |  +- org.ops4j.base:ops4j-base-net:jar:1.5.1:test
[INFO] |  |  \- org.ops4j.pax.swissbox:pax-swissbox-bnd:jar:1.8.5:test
[INFO] |  |     \- biz.aQute.bnd:biz.aQute.bndlib:jar:6.1.0:test
[INFO] |  |        +- org.osgi:org.osgi.dto:jar:1.0.0:test
[INFO] |  |        +- org.osgi:org.osgi.resource:jar:1.0.0:test
[INFO] |  |        +- org.osgi:org.osgi.util.tracker:jar:1.5.4:provided
[INFO] |  |        |  \- org.osgi:osgi.annotation:jar:8.0.1:provided
[INFO] |  |        +- org.osgi:org.osgi.service.log:jar:1.5.0:provided
[INFO] |  |        +- org.osgi:org.osgi.service.repository:jar:1.1.0:provided
[INFO] |  |        +- org.osgi:org.osgi.util.function:jar:1.2.0:provided
[INFO] |  |        +- org.osgi:org.osgi.util.promise:jar:1.2.0:provided
[INFO] |  |        \- biz.aQute.bnd:biz.aQute.bnd.util:jar:6.1.0:test
[INFO] |  +- com.guicedee.services:javax.inject:jar:1.0.20.0:provided
[INFO] |  +- org.osgi:org.osgi.framework:jar:1.10.0:provided
[INFO] |  +- org.opendaylight.odlparent:opendaylight-karaf-empty:zip:12.0.4:test
[INFO] |  |  +- org.apache.karaf.features:framework:kar:4.4.3:test
[INFO] |  |  |  +- org.apache.karaf.features:base:jar:4.4.3:test
[INFO] |  |  |  |  +- org.apache.servicemix.specs:org.apache.servicemix.specs.activation-api-1.2.1:jar:1.2.1_3:test
[INFO] |  |  |  |  +- org.apache.karaf.specs:org.apache.karaf.specs.locator:jar:4.4.3:test
[INFO] |  |  |  |  +- org.apache.karaf.specs:org.apache.karaf.specs.java.xml:jar:4.4.3:test
[INFO] |  |  |  |  |  +- org.apache.geronimo.specs:geronimo-stax-api_1.2_spec:jar:1.2:test
[INFO] |  |  |  |  |  \- xml-apis:xml-apis:jar:1.4.01:test
[INFO] |  |  |  |  +- org.apache.karaf.specs:org.apache.karaf.specs.java.xml.ws:jar:4.4.3:test
[INFO] |  |  |  |  |  +- org.apache.geronimo.specs:geronimo-saaj_1.3_spec:jar:1.1:test
[INFO] |  |  |  |  |  \- org.apache.geronimo.specs:geronimo-jaxws_2.2_spec:jar:1.2:test
[INFO] |  |  |  |  \- org.apache.karaf.specs:org.apache.karaf.specs.activator:jar:4.4.3:test
[INFO] |  |  |  +- org.apache.karaf:org.apache.karaf.main:jar:4.4.3:test
[INFO] |  |  |  |  +- org.apache.karaf:org.apache.karaf.util:jar:4.4.3:test
[INFO] |  |  |  |  |  \- org.apache.felix:org.apache.felix.utils:jar:1.11.8:test
[INFO] |  |  |  |  +- net.java.dev.jna:jna:jar:5.12.1:test
[INFO] |  |  |  |  \- net.java.dev.jna:jna-platform:jar:5.12.1:test
[INFO] |  |  |  +- org.apache.karaf:org.apache.karaf.client:jar:4.4.3:test
[INFO] |  |  |  |  \- org.apache.sshd:sshd-osgi:jar:2.9.2:test
[INFO] |  |  |  +- org.apache.karaf.jaas:org.apache.karaf.jaas.boot:jar:4.4.3:test
[INFO] |  |  |  +- org.apache.karaf.diagnostic:org.apache.karaf.diagnostic.boot:jar:4.4.3:test
[INFO] |  |  |  +- org.apache.karaf.config:org.apache.karaf.config.core:jar:4.4.3:test
[INFO] |  |  |  |  +- org.apache.felix:org.apache.felix.cm.json:jar:1.0.6:test
[INFO] |  |  |  |  +- org.apache.sling:org.apache.sling.commons.johnzon:jar:1.2.14:test
[INFO] |  |  |  |  |  \- org.osgi:org.osgi.annotation.versioning:jar:1.1.2:provided
[INFO] |  |  |  |  \- org.apache.geronimo.specs:geronimo-json_1.1_spec:jar:1.5:test
[INFO] |  |  |  +- org.eclipse.platform:org.eclipse.osgi:jar:3.18.0:test
[INFO] |  |  |  +- org.apache.felix:org.apache.felix.framework:jar:7.0.5:test
[INFO] |  |  |  |  \- org.codehaus.mojo:animal-sniffer-annotations:jar:1.9:test
[INFO] |  |  |  +- org.jline:jline:jar:3.21.0:test
[INFO] |  |  |  +- org.ops4j.pax.logging:pax-logging-api:jar:2.2.0:test
[INFO] |  |  |  |  +- org.osgi:osgi.core:jar:8.0.0:provided
[INFO] |  |  |  |  \- org.osgi:osgi.cmpn:jar:7.0.0:test
[INFO] |  |  |  +- org.ops4j.pax.logging:pax-logging-log4j2:jar:2.2.0:test
[INFO] |  |  |  +- org.ops4j.pax.logging:pax-logging-logback:jar:2.2.0:test
[INFO] |  |  |  |  +- ch.qos.logback:logback-core:jar:1.2.11:test
[INFO] |  |  |  |  +- ch.qos.logback:logback-classic:jar:1.2.11:test
[INFO] |  |  |  |  +- ch.qos.logback.contrib:logback-jackson:jar:0.1.5:test
[INFO] |  |  |  |  +- ch.qos.logback.contrib:logback-json-core:jar:0.1.5:test
[INFO] |  |  |  |  \- ch.qos.logback.contrib:logback-json-classic:jar:0.1.5:test
[INFO] |  |  |  +- org.apache.felix:org.apache.felix.fileinstall:jar:3.7.4:test
[INFO] |  |  |  +- org.apache.felix:org.apache.felix.configadmin.plugin.interpolation:jar:1.2.6:test
[INFO] |  |  |  \- org.apache.karaf.features:org.apache.karaf.features.extension:jar:4.4.3:test
[INFO] |  |  +- org.apache.karaf.features:framework:xml:features:4.4.3:test
[INFO] |  |  +- org.apache.aries.quiesce:org.apache.aries.quiesce.api:jar:1.0.0:test
[INFO] |  |  +- org.apache.felix:org.apache.felix.metatype:jar:1.2.4:test
[INFO] |  |  +- org.opendaylight.odlparent:karaf.branding:jar:12.0.4:test
[INFO] |  |  +- org.opendaylight.odlparent:bcpkix-framework-ext:jar:12.0.4:test
[INFO] |  |  +- org.opendaylight.odlparent:bcprov-framework-ext:jar:12.0.4:test
[INFO] |  |  \- org.opendaylight.odlparent:bcutil-framework-ext:jar:12.0.4:test
[INFO] |  \- org.opendaylight.odlparent:karaf-util:jar:12.0.4:test
[INFO] |     +- org.apache.maven.wagon:wagon-http:jar:3.5.2:test
[INFO] |     |  +- org.apache.maven.wagon:wagon-http-shared:jar:3.5.2:test
[INFO] |     |  +- org.apache.httpcomponents:httpclient:jar:4.5.13:test
[INFO] |     |  |  \- commons-codec:commons-codec:jar:1.11:test
[INFO] |     |  +- org.apache.httpcomponents:httpcore:jar:4.4.15:test
[INFO] |     |  +- org.codehaus.plexus:plexus-utils:jar:3.3.0:test
[INFO] |     |  \- org.apache.maven.wagon:wagon-provider-api:jar:3.5.2:test
[INFO] |     \- org.osgi:org.osgi.service.url:jar:1.0.1:provided
[INFO] +- org.opendaylight.odlparent:bundles-test-lib:jar:12.0.4:test
[INFO] |  \- org.awaitility:awaitility:jar:4.1.1:test
[INFO] +- org.apache.karaf.bundle:org.apache.karaf.bundle.core:jar:4.4.3:test
[INFO] +- org.slf4j:slf4j-api:jar:1.7.32:test
[INFO] +- org.eclipse.jdt:org.eclipse.jdt.annotation:jar:2.2.700:provided
[INFO] +- com.github.spotbugs:spotbugs-annotations:jar:4.7.3:provided
[INFO] +- org.gaul:modernizer-maven-annotations:jar:2.6.0:provided
[INFO] +- org.slf4j:slf4j-simple:jar:1.7.32:test
[INFO] +- org.hamcrest:hamcrest:jar:2.2:test
[INFO] +- org.hamcrest:hamcrest-library:jar:2.2:test
[INFO] |  \- org.hamcrest:hamcrest-core:jar:2.2:test
[INFO] +- org.junit.jupiter:junit-jupiter-api:jar:5.9.2:test
[INFO] |  +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO] |  +- org.junit.platform:junit-platform-commons:jar:1.9.2:test
[INFO] |  \- org.apiguardian:apiguardian-api:jar:1.1.2:test
[INFO] +- org.junit.jupiter:junit-jupiter-params:jar:5.9.2:test
[INFO] +- org.junit.jupiter:junit-jupiter-engine:jar:5.9.2:test
[INFO] |  \- org.junit.platform:junit-platform-engine:jar:1.9.2:test
[INFO] +- org.mockito:mockito-junit-jupiter:jar:5.2.0:test
[INFO] +- junit:junit:jar:4.13.2:test
[INFO] +- org.junit.jupiter:junit-jupiter-migrationsupport:jar:5.9.2:test
[INFO] +- org.junit.vintage:junit-vintage-engine:jar:5.9.2:test
[INFO] \- org.mockito:mockito-core:jar:5.2.0:test
[INFO]    +- net.bytebuddy:byte-buddy:jar:1.14.1:compile
[INFO]    +- net.bytebuddy:byte-buddy-agent:jar:1.14.1:test
[INFO]    \- org.objenesis:objenesis:jar:3.3:test
hboutemy commented 1 year ago

IIUC, it's https://github.com/opendaylight/aaa ?

rovarga commented 1 year ago

IIUC, it's https://github.com/opendaylight/aaa ?

correct, it is currently using 2.7.5. Attempting to bump to 2.7.6 (via https://git.opendaylight.org/gerrit/c/aaa/+/105524) ends up exceeding 60 minutes build time, and thus fails to verify with:

23:45:14  [INFO] --- cyclonedx-maven-plugin:2.7.6:makeBom (default) @ aaa-karaf ---
23:45:14  [INFO] CycloneDX: Resolving Dependencies
23:52:04  Build timed out (after 60 minutes). Marking the build as failed.

I have verified locally 2.7.7 behaves essentially the same.

hboutemy commented 1 year ago

@knrc Aether / Maven Resolver API is much more chatty (many more dependency nodes) than previous Maven Dependency Graph: we'll need to optimize to cut the dependency graph when it's useful... do you have any known improvements in your WIP?

knrc commented 1 year ago

@hboutemy Chatty in what way? IIRC it should be the same nodes as the maven one (different type obviously) but with direct access to the winner information hidden by the maven tree.

knrc commented 1 year ago

I can take a look at this and see what is going on

knrc commented 1 year ago

@hboutemy initial suspicion is that this is related to correctly identifying the clashes and #311, I'll dig further into this and work on #311 next

knrc commented 1 year ago

@rovarga @hboutemy I've rebased my changes for #311 onto master and submitted a PR, see #345

I'm seeing the following results with the odl-aaa-encryption-service makeBom.

2.7.5

Total time:  10.076 s
real    0m11.686s
user    0m34.832s
sys 0m2.923s

2.7.6

Total time:  01:32 min
real    1m33.594s
user    2m3.219s
sys 0m3.329s

2.7.7

Total time:  01:27 min
real    1m28.759s
user    2m1.641s
sys 0m3.301s

2.7.8-SNAPSHOT

Total time:  9.354 s
real    0m10.526s
user    0m41.266s
sys 0m2.697s

@rovarga can you please try this locally and see how it performs for you?

rovarga commented 1 year ago

@rovarga can you please try this locally and see how it performs for you?

I can confirm this fixes the performance issue. Tracing also indicates there are only 2577 invocations of PackageURL.canonicalize().

Thanks for the quick turnaround! :)

knrc commented 1 year ago

@rovarga Thanks for the confirmation. There's one thing left for me to do with that PR, not related to your use case, so I'll try to get it completed as quickly as I can.

hboutemy commented 1 year ago

fixed by #345