Import of CycloneDX Scan fails if description of CVE is missing

uyless commented 8 months ago

Bug description

Hey,

we are trying to implement a process in which we scan all our docker images with trivy. The resulting CycloneDX should be send to DefectDojo. Some Images had a CVE such as https://security-tracker.debian.org/tracker/TEMP-0290435-0B57B5. This CVE did not have a description in the beginning, after a few days debian added one. Anyways, in DefectDojo the description field seems to be mandatory but the specification of CycloneDX does not state any requirements on the fields of the vulnerability objects https://cyclonedx.org/docs/1.5/json/#vulnerabilities_items_description. We were not able to import those CycloneDX SBOMs using the API /api/v2/reimport-scan/

Thats beeing said I think such a requirement should also not be expected on CycloneDX SBOM imports.

Steps to reproduce

Create SBOM using trivy

trivy image --format cyclonedx --cache-dir .trivycache/ --scanners vuln --no-progress <some docker image>

Add to vulnerabilities list:

{
  "id": "TEMP-0290435-0B57B5",
  "source": {
    "name": "debian",
    "url": "https://salsa.debian.org/security-tracker-team/security-tracker"
  },
  "ratings": [
    {
      "source": {
        "name": "debian"
      },
      "severity": "low"
    }
  ],
  "advisories": [
    {
      "url": "https://security-tracker.debian.org/tracker/TEMP-0290435-0B57B5"
    }
  ],
  "affects": [
    {
      "ref": "pkg:deb/debian/tar@1.30%2Bdfsg-6?arch=amd64&distro=debian-10.12",
      "versions": [
        {
          "version": "1.30+dfsg-6",
          "status": "affected"
        }
      ]
    }
  ]
},

Send sbom to defect-dojo using reimport or import api endpoint:

curl -v -X POST --header "X-CSRFToken: $TOKEN" -H 'Content-Type: multipart/form-data' --header "Authorization: Token $TOKEN" \
          -F "active=true" \
          -F "verified=true" \
          -F "scan_type=CycloneDX Scan" \
          -F "product_name=test_product" \
          -F "engagement_name=test_engangement" \
          -F "product_type_name=test_type" \
          -F "auto_create_context=true" \
          -F "file=@sbom_file" \
'https://<defect-dojo-host>/api/v2/reimport-scan/'

Check in server logs for 'django.db.utils.IntegrityError: null value in column "description" violates not-null constraint'

Expected behavior Import should be possible without giving a description in a vulnerability entry, at least since specification does not mention any requirements on this key.

Deployment method (select with an X)

[ ] Docker Compose
[x] Kubernetes
[ ] GoDojo

Environment information

Operating System: Ubuntu 22.04
DefectDojo version 2.29.4

Logs

[04/Jan/2024 15:12:59] ERROR [dojo.api_v2.exception_handler:56] null value in column "description" violates not-null constraint
DETAIL:  Failing row contains (26640, tar:1.30+dfsg-6 | TEMP-0290435-0B57B5, 2024-01-04, 0, null, Low, null, null, null, null, null, **URL:** https://security-tracker.debian.org/tracker/TEMP-029043..., t, t, f, f, f, f, f, 0, null, S3, 2024-01-04 15:12:59.112358+00, null, null, null, null, null, t, f, 2024-01-04 15:12:59.42694+00, null, null, null, 7, null, 7, null, 162, null, f, null, null, null, null, null, null, null, tar, 1.30+dfsg-6, null, TEMP-0290435-0B57B5, f, null, 2024-01-04 15:12:59.426904+00, null, null, null, null, null, null).
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.instana/python/instana/instrumentation/pep0249.py", line 51, in execute
    result = self.__wrapped__.execute(sql, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.NotNullViolation: null value in column "description" violates not-null constraint
DETAIL:  Failing row contains (26640, tar:1.30+dfsg-6 | TEMP-0290435-0B57B5, 2024-01-04, 0, null, Low, null, null, null, null, null, **URL:** https://security-tracker.debian.org/tracker/TEMP-029043..., t, t, f, f, f, f, f, 0, null, S3, 2024-01-04 15:12:59.112358+00, null, null, null, null, null, t, f, 2024-01-04 15:12:59.42694+00, null, null, null, 7, null, 7, null, 162, null, f, null, null, null, null, null, null, null, tar, 1.30+dfsg-6, null, TEMP-0290435-0B57B5, f, null, 2024-01-04 15:12:59.426904+00, null, null, null, null, null, null).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/rest_framework/mixins.py", line 19, in create
    self.perform_create(serializer)
  File "/app/dojo/api_v2/views.py", line 3472, in perform_create
    serializer.save(push_to_jira=push_to_jira)
  File "/app/dojo/api_v2/serializers.py", line 2548, in save
    ) = importer.import_scan(
        ^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/importers/importer/importer.py", line 345, in import_scan
    new_findings = self.process_parsed_findings(test, parsed_findings, scan_type, user, active=active,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/decorators.py", line 48, in __wrapper__
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/celery/local.py", line 182, in __call__
    return self._get_current_object()(*a, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/celery/app/task.py", line 411, in __call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/importers/importer/importer.py", line 111, in process_parsed_findings
    item.save(dedupe_option=False)
  File "/app/dojo/models.py", line 2937, in save
    super(Finding, self).save(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 812, in save
    self.save_base(
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 863, in save_base
    updated = self._save_table(
              ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1006, in _save_table
    results = self._do_insert(
              ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1047, in _do_insert
    return manager._insert(
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1791, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1660, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.instana/python/instana/instrumentation/pep0249.py", line 51, in execute
    result = self.__wrapped__.execute(sql, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.IntegrityError: null value in column "description" violates not-null constraint
DETAIL:  Failing row contains (26640, tar:1.30+dfsg-6 | TEMP-0290435-0B57B5, 2024-01-04, 0, null, Low, null, null, null, null, null, **URL:** https://security-tracker.debian.org/tracker/TEMP-029043..., t, t, f, f, f, f, f, 0, null, S3, 2024-01-04 15:12:59.112358+00, null, null, null, null, null, t, f, 2024-01-04 15:12:59.42694+00, null, null, null, 7, null, 7, null, 162, null, f, null, null, null, null, null, null, null, tar, 1.30+dfsg-6, null, TEMP-0290435-0B57B5, f, null, 2024-01-04 15:12:59.426904+00, null, null, null, null, null, null).

Sample scan files Thats just a scan with trivy from ubuntu:latest and I added a vulnerability without description. ubuntu.json

Additional context (optional) First I was thinking to raise an issue at trivy but it seems wrong that the specification of CycloneDX does not mention a requirement on this field.

manuel-sommer commented 8 months ago

This can be closed @mtesauro

k0mand1r commented 4 months ago

Also reported in: https://github.com/DefectDojo/django-DefectDojo/issues/10249

We still experience this bug in version 2.34.4. The input is a CyclonDX report which does not generate a description in the XML format.

16/May/2024 13:22:55] ERROR [dojo.api_v2.exception_handler:36] null value in column "description" of relation "dojo_finding" violates not-null constraint

We've concluded that after looking at the codebase that there are 2 xml parser functions (legacy for CycloneDX 1.0, and a new on for a higher CDX version). This fix has only been merged on the legacy function after the refactor earlier this year. As we're using CycloneDX version 1.4+ the issue still persists.

manage_vulnerabilty_xml() (New function)
manage_vulnerability_legacy() (Legacy function)

manuel-sommer commented 4 months ago

Hi @k0mand1r,

could you please add a sample file? Than, I can submit a bugfix.

Best

mtesauro commented 4 months ago

@k0mand1r It's also quite possible that whatever tool generated that XML file isn't valid per the CycloneDX spec - this wouldn't be the first time a tool produced "ClycloneDX-like" output that didn't pass a validation test.

I'd suggest you use the cyclonedx tool to validate your XML files to ensure they are actually valid (and should be parsed by DefectDojo's parser). If it's valid and DefectDojo can't parse it, then there's definitely an issue with our parser. If it's not valid ClycloneDX, then that's on the tool that output the XML file.

You can get the CLI tool at: https://github.com/CycloneDX/cyclonedx-cli

I've used it, it works great.

DefectDojo / django-DefectDojo

Import of CycloneDX Scan fails if description of CVE is missing #9277