Fails to pick up license information

black-snow commented 1 month ago

Current Behavior

For some projects DT fails to pick up license information from the cyclonedx report, yielding a lot of false positives.

Example excerpt:

{
  "$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "serialNumber": "urn:uuid:82c0b550-139d-4f9d-9066-6e051d3796e3",
  "version": 1,
  "metadata": {
    "timestamp": "2024-05-30T07:31:19+00:00",
    "tools": {
      "components": [
        {
          "type": "application",
          "group": "aquasecurity",
          "name": "trivy",
          "version": "0.51.4"
        }
      ]
    },
...
    {
      "bom-ref": "pkg:pypi/wsproto@1.2.0",
      "type": "library",
      "name": "wsproto",
      "version": "1.2.0",
      "licenses": [
        {
          "license": {
            "name": "MIT License"
          }
        }
      ],
      "purl": "pkg:pypi/wsproto@1.2.0",
      "properties": [
        {
          "name": "aquasecurity:trivy:FilePath",
          "value": "home/seluser/.local/lib/python3.10/site-packages/wsproto-1.2.0.dist-info/METADATA"
        },
        {
          "name": "aquasecurity:trivy:LayerDiffID",
          "value": "sha256:dd3dae5c3fc77d954f222600681c0e2844cb3a1fa33de9c9602a6f99e2bd9e83"
        },
        {
          "name": "aquasecurity:trivy:PkgType",
          "value": "python-pkg"
        }
      ]
    }
...

wsproto ends up in DT 4.11.1 with no license information. I turned on BOM validation an there seem to be no issues:

2024-05-30 07:46:49,317 INFO [BomUploadProcessingTask] Processing CycloneDX BOM uploaded to project: 229e7e71-3ded-467f-9bb4-f8f7a705d7f3 2024-05-30 07:47:31,523 INFO [BomUploadProcessingTask] Identified 0 new components 2024-05-30 07:47:31,523 INFO [BomUploadProcessingTask] Processing CycloneDX dependency graph for project: 229e7e71-3ded-467f-9bb4-f8f7a705d7f3 2024-05-30 07:47:31,786 INFO [BomUploadProcessingTask] Processed 565 components and 0 services uploaded to project 229e7e71-3ded-467f-9bb4-f8f7a705d7f3 2024-05-30 07:47:31,787 INFO [RepositoryMetaAnalyzerTask] Performing component repository metadata analysis against 565 components 2024-05-30 07:47:33,414 INFO [InternalAnalysisTask] Starting internal analysis task 2024-05-30 07:47:33,414 INFO [InternalAnalysisTask] Analyzing 564 component(s) 2024-05-30 07:47:34,025 INFO [RepositoryMetaAnalyzerTask] Completed component repository metadata analysis against 565 components 2024-05-30 07:47:34,026 INFO [PolicyEngine] Evaluating 565 component(s) against applicable policies 2024-05-30 07:47:39,498 INFO [InternalAnalysisTask] Internal analysis complete 2024-05-30 07:47:39,505 WARN [OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access 2024-05-30 07:47:39,508 INFO [OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task 2024-05-30 07:47:43,543 INFO [OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2024-05-30 07:47:43,547 INFO [PolicyEngine] Evaluating 565 component(s) against applicable policies 2024-05-30 07:47:50,135 INFO [PolicyEngine] Policy analysis complete 2024-05-30 07:47:50,136 INFO [ProjectMetricsUpdateTask] Executing metrics update for project 229e7e71-3ded-467f-9bb4-f8f7a705d7f3 2024-05-30 07:47:58,565 INFO [PolicyEngine] Policy analysis complete 2024-05-30 07:47:58,565 INFO [ProjectMetricsUpdateTask] Executing metrics update for project 229e7e71-3ded-467f-9bb4-f8f7a705d7f3

Steps to Reproduce

POST above SBOM to DT.
check wsproto component license information

Expected Behavior

wsproto should have MIT License set.

Dependency-Track Version

4.11.1

Dependency-Track Distribution

Container Image

Database Server

PostgreSQL

Database Server Version

16.2

Browser

Mozilla Firefox

Checklist

[X] I have read and understand the contributing guidelines
[X] I have checked the existing issues for whether this defect was already reported

black-snow commented 1 month ago

Hi @nscuro , with 4.11.2 it now fails to import anythig:

2024-06-03 08:25:42,449 INFO [BomUploadProcessingTask] Processing CycloneDX BOM uploaded to project: 56d254dc-e764-456a-944a-c496f531712d
2024-06-03 08:25:42,934 ERROR [Persist] Insert of object "org.dependencytrack.model.License@5fee4b91" using statement "INSERT INTO "LICENSE" ("COMMENT","ISCUSTOMLICENSE","ISDEPRECATED","FSFLIBRE","HEADER","LICENSEID","NAME","ISOSIAPPROVED","SEEALSO","TEMPLATE","TEXT","UUID") VALUES (?,?,?,?,?,?,?,?,?,?,?,?)" failed : ERROR: null value in column "NAME" of relation "LICENSE" violates not-null constraint
  Detail: Failing row contains (705, null, f, f, f, null, null, null, f, null, null, null, 4b655bac-26a7-40f3-889c-dd86e765f55a).
2024-06-03 08:25:42,936 ERROR [BomUploadProcessingTask] Error while processing bom
org.datanucleus.exceptions.NucleusDataStoreException: Insert of object "org.dependencytrack.model.License@5fee4b91" using statement "INSERT INTO "LICENSE" ("COMMENT","ISCUSTOMLICENSE","ISDEPRECATED","FSFLIBRE","HEADER","LICENSEID","NAME","ISOSIAPPROVED","SEEALSO","TEMPLATE","TEXT","UUID") VALUES (?,?,?,?,?,?,?,?,?,?,?,?)" failed : ERROR: null value in column "NAME" of relation "LICENSE" violates not-null constraint
  Detail: Failing row contains (705, null, f, f, f, null, null, null, f, null, null, null, 4b655bac-26a7-40f3-889c-dd86e765f55a).
    at org.datanucleus.store.rdbms.request.RequestUtil.convertSqlException(RequestUtil.java:41)
    at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:625)
    at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObjectInTable(RDBMSPersistenceHandler.java:235)
    at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:211)
    at org.datanucleus.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:4614)
    at org.datanucleus.state.StateManagerImpl.makePersistent(StateManagerImpl.java:4591)
    at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2076)
    at org.datanucleus.ExecutionContext.persistObjectInternal(ExecutionContext.java:320)
    at org.datanucleus.store.rdbms.mapping.java.PersistableMapping.setObjectAsValue(PersistableMapping.java:632)
    at org.datanucleus.store.rdbms.mapping.java.PersistableMapping.setObject(PersistableMapping.java:381)
    at org.datanucleus.store.rdbms.fieldmanager.ParameterSetter.storeObjectField(ParameterSetter.java:191)
    at org.datanucleus.state.StateManagerImpl.providedObjectField(StateManagerImpl.java:1939)
    at org.dependencytrack.model.Component.dnProvideField(Component.java)
    at org.dependencytrack.model.Component.dnProvideFields(Component.java)
    at org.datanucleus.state.StateManagerImpl.provideFields(StateManagerImpl.java:2583)
    at org.datanucleus.store.rdbms.request.UpdateRequest.execute(UpdateRequest.java:436)
    at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.updateObjectInTable(RDBMSPersistenceHandler.java:529)
    at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.updateObject(RDBMSPersistenceHandler.java:494)
    at org.datanucleus.state.StateManagerImpl.flush(StateManagerImpl.java:5917)
    at org.datanucleus.flush.FlushOrdered.execute(FlushOrdered.java:96)
    at org.datanucleus.ExecutionContextImpl.flushInternal(ExecutionContextImpl.java:4050)
    at org.datanucleus.ExecutionContextImpl.processNontransactionalAtomicChanges(ExecutionContextImpl.java:1473)
    at org.datanucleus.ExecutionContextImpl.processNontransactionalUpdate(ExecutionContextImpl.java:1434)
    at org.datanucleus.state.StateManagerImpl.setObjectField(StateManagerImpl.java:3224)
    at org.dependencytrack.model.Component.dnSetresolvedLicense(Component.java)
    at org.dependencytrack.model.Component.setResolvedLicense(Component.java:678)
    at org.dependencytrack.parser.cyclonedx.util.ModelConverter.convert(ModelConverter.java:574)
    at org.dependencytrack.parser.cyclonedx.util.ModelConverter.convertComponents(ModelConverter.java:462)
    at org.dependencytrack.tasks.BomUploadProcessingTask.inform(BomUploadProcessingTask.java:157)
    at org.dependencytrack.tasks.BomUploadProcessingTaskV2.inform(BomUploadProcessingTaskV2.java:151)
    at alpine.event.framework.BaseEventService.lambda$publish$0(BaseEventService.java:110)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "NAME" of relation "LICENSE" violates not-null constraint
  Detail: Failing row contains (705, null, f, f, f, null, null, null, f, null, null, null, 4b655bac-26a7-40f3-889c-dd86e765f55a).
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2725)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2412)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:371)
    at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:502)
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:419)
    at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:194)
    at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:155)
    at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
    at org.datanucleus.store.rdbms.SQLController.doExecuteStatementUpdate(SQLController.java:463)
    at org.datanucleus.store.rdbms.SQLController.executeStatementUpdateDeferRowCountCheckForBatching(SQLController.java:413)
    at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:532)
    ... 32 common frames omitted

black-snow commented 1 month ago

Small correction, it did import a handful of components but is about a couple hundred short.

black-snow commented 1 month ago

ref.: https://github.com/DependencyTrack/dependency-track/issues/3798