Vulnerability with Unassigned Severity

melba-lopez commented 1 year ago

Current Behavior

I've been seeing some odd issues with an instance of DT lately, where previously imported CVEs with appropriate CVSS Scores are no longer reporting them.

I went through the logs of dependency track and validated that it was consistently getting data properly for the nvd feeds. I went as far as downloading the actual file and validating in the json that the base, severity,and exploitability score were there. After further investigating in another instance of DT (Env B), I noticed that it is backlevel and only has NVD enabled. It is not having these issues and is reporting CVEs properly.

Env A: 4.6.3 DT + OSS Index + NVD + GH Advisories + OSV

Here's an example where the same vulnerability (even with aliasing to eachother) has one showing up as "unassigned severity"

Env B: 4.6.1 DT + NVD

Steps to Reproduce

4.6.3 container (with separate api, db, front end)
Let Internal analyzer/nvd settle
Import any bom to a project
an hour or two after project is reporting vulns from NVD, enable github advisories, google OSV, and OSS Index

Expected Behavior

Regardless of how many analyzers we enable, DT should be able to

Consistently report severity of vulnerabilities (especially when aliased to eachother)
A properly reported CVE should not go into an unassigned mode (when nothing about the CVE has changed)
If there are discrepancies, it would be good to at least modify the severity to match an existing alias.

Dependency-Track Version

4.6.2

Dependency-Track Distribution

Container Image

Database Server

PostgreSQL

Database Server Version

13

Browser

N/A

Checklist

[X] I have read and understand the contributing guidelines
[X] I have checked the existing issues for whether this defect was already reported

melba-lopez commented 1 year ago

I just installed a local copy of Dependency Track v4.7 on my system (pre-built bundled container image) and replicated the same issue. Let the system settle with an sbom and its analysis...then populate Github Adv, Google OSV, and Sonatype. After it starts with the new analysis, some vulns go from high/critical to unassigned.

melba-lopez commented 1 year ago

Here is a sample package that went from all the vulnerabilities assigned (with NVD properly populated in the database) to the majority of them going into an unassigned mode.

Here's an example of a vulnerability identified by GitHub, with the CVSS3.0 severity listed on NVD. However, when you go to the NVD source (which used to have appropriate severity, it no longer has it.

Do you think its this code commit https://github.com/DependencyTrack/dependency-track/commit/96268b79e7d451cf20800689e42ae67a694d3d0a for src/main/java/org/dependencytrack/util/VulnerabilityUtil.java that is causing the issue? Perhaps the issue is with the new code logic for "severity assignment."

Looks like it just changes anything with an owasp risk rating of unassigned to Unassigned severity in NVD. Not sure why NVD info is being overwritten, but its definitely happening when you enable multiple sources of analyzers/vuln sources.

syalioune commented 1 year ago

Hey @melba-lopez , I'll look into it.

syalioune commented 1 year ago

FYI the commit you're referencing was only introduced in 4.7 and since your issue date back from 4.6.2, the source should be elsewhere.

melba-lopez commented 1 year ago

thanks @syalioune! First time looking at the actual code and trying to see where it could possibly be happening. I did not notice the versioning for the commit, so thanks for clarifying.

melba-lopez commented 1 year ago

I did a fresh install and I turned on the analyzers/vuln sources one by one. The only one I haven't turned on as of yet is GitHub Advisories. The number of unassigned has not changed, but vulns have gone up. So potentially its the GitHub Advisories source thats causing it to overwrite NVD data.

syalioune commented 1 year ago

Hello @melba-lopez,

I wasn't able to reproduce :confused: and I don't see any path in the code where NVD data could be overwritten as you described (except from the NVD data mirroring).

Some questions and some clarifications below to further the analysis

Questions

Your database state seems a bit off to me. In the screenshot for vulnerability GHSA-7534-mm45-c74v, there are two aliases to CVE-2021-34552. Same thing, when looking on the CVE side. There should be only one. Do you happen to reproduce those duplicates on every fresh install ?
By fresh install, do you mean an installation with a new/different database ? If that is the case, are you using Postgresql or H2 for those fresh installs
This part confuses me Here's an example of a vulnerability identified by GitHub, with the CVSS3.0 severity listed on NVD. However, when you go to the NVD source (which used to have appropriate severity, it no longer has it. Does the provided screenshot of the CVSS3.0 vector come from Github source details screen or NVD source details screen ?
Can you execute the SQL requests below at every step of the procedure (nominal and before enabling each analyzer) and give back the results ? You may adapt the identifier based on what you observe

select * from vulnerability where vulnid in ('CVE-2021-34552','GHSA-7534-mm45-c74v');
select * from vulnerabilityalias where cve_id = 'CVE-2021-34552' or ghsa_id = 'GHSA-7534-mm45-c74v';

Clarifications

Consistently report severity of vulnerabilities (especially when aliased to eachother) && If there are discrepancies, it would be good to at least modify the severity to match an existing alias.

That is an entirely issue (enhancement) altogether. Current implementation only provide a way to link similar vulnerabilities based on vulnerability intelligence data. Dependency Track does not correlate severity among aliases (see https://github.com/DependencyTrack/dependency-track/issues/1642). I kindly suggest to create a distinct issue to log that specific enhancement request.

melba-lopez commented 1 year ago

Hi @syalioune all great questions!

The Pillow "fresh install" example and the screenshots in that post was green field (no volumes no containers existed before) on my laptop of 4.7.0 using these 2 commands posted on the website: https://dependencytrack.org/#download

I see the same behaviour on the 4.6.3 DT Env with Sep DB, but what is more troubling is that you see it try to correct itself every so often (i.e. NVD severities may be updated in the db) but then it immediately goes to unassigned (see screenshot below). Again weird behaviour, but I've managed to replicate it in 3 different environments with 3 different versions and 3 different deployment configurations.

I will try to answer the remaining this evening. (Hoping to do a recording showing a fresh-install and replicate in the video)

melba-lopez commented 1 year ago

Clarifications

That is an entirely issue (enhancement) altogether. Current implementation only provide a way to link similar vulnerabilities based on vulnerability intelligence data. Dependency Track does not correlate severity among aliases (see #1642). I kindly suggest to create a distinct issue to log that specific enhancement request.

Will do!

This part confuses me Here's an example of a vulnerability identified by GitHub, with the CVSS3.0 severity listed on NVD. However, when you go to the NVD source (which used to have appropriate severity, it no longer has it. Does the provided screenshot of the CVSS3.0 vector come from Github source details screen or NVD source details screen ?

Can you execute the SQL requests below at every step of the procedure (nominal and before enabling each analyzer) and give back the results ? You may adapt the identifier based on what you observe

I destroyed the instance when i did a new deployment so I can't provide answers for these :(

REPRODUCED 2X via GoogleOSV (see details below)

I mistakenly thought it was GitHub Adivsories because of the aliasing. However, it looks like it. happens when Google OSV is enabled (which I know is in Beta --- so I def won't turn it on again ;-) ). But, its still odd that it overrides NVD data. I've taken 3 videos.

Each video has its own label of which "deployment" and what is enabled.

syalioune commented 1 year ago

Thanks for the video, it was super helpful 👍 Made me realize my attempts at reproducing the issue were half complete : I only enabled PyPI ecosystem for OSV whereas the issue is coming from Alpine:3.15 ecosystem.

You're indeed right, the issue that you observe are all due to OSV integration which could override NVD data (actually Github Advisory data could also be overriden).

The vulnerability update logic should be updated to not erase existing data if the vulnerability source is missing some attributes

Going into details with CVE-2021-34552 as example :

Vulnerability introduction through OSS Index analysis - Severity is OK

curl --location --request POST 'https://ossindex.sonatype.org/api/v3/component-report' --header 'Authorization: Basic xyz' --header 'Content-Type: application/json' --data-raw '{"coordinates": ["pkg:pypi/pillow@7.0.0"]}'

[
   {
      "coordinates":"pkg:pypi/pillow@7.0.0",
      "reference":"https://ossindex.sonatype.org/component/pkg:pypi/pillow@7.0.0?utm_source=postmanruntime&utm_medium=integration&utm_content=7.30.0",
      "vulnerabilities":[
         {
            "id":"CVE-2021-34552",
            "displayName":"CVE-2021-34552",
            "title":"[CVE-2021-34552]CWE-120:BufferCopywithoutCheckingSizeofInput('ClassicBufferOverflow')",
            "description":"Pillowthrough8.2.0andPIL(akaPythonImagingLibrary)through1.1.7allowanattackertopasscontrolledparametersdirectlyintoaconvertfunctiontotriggerabufferoverflowinConvert.c.",
            "cvssScore":9.8,
            "cvssVector":"CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H",
            "cwe":"CWE-120",
            "cve":"CVE-2021-34552",
            "reference":"https://ossindex.sonatype.org/vulnerability/CVE-2021-34552?component-type=pypi&component-name=pillow&utm_source=postmanruntime&utm_medium=integration&utm_content=7.30.0",
            "externalReferences":[
               "http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2021-34552",
               "https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html#buffer-overflow",
               "https://github.com/python-pillow/Pillow/commit/d254e581a1f97c52b3796ddb48d09b48bd16d446",
               "https://github.com/python-pillow/Pillow/pull/5567"
            ]
         }
      ]
   }
]

Vulnerability update through NVD parsing - Severity is OK

{
   "cve":{
      "data_type":"CVE",
      "data_format":"MITRE",
      "data_version":"4.0",
      "CVE_data_meta":{
         "ID":"CVE-2021-34552",
         "ASSIGNER":"cve@mitre.org"
      },
      "problemtype":{
         "problemtype_data":[
            {
               "description":[
                  {
                     "lang":"en",
                     "value":"CWE-120"
                  }
               ]
            }
         ]
      },
      "references":{
         "reference_data":[
            {
               "url":"https://pillow.readthedocs.io/en/stable/releasenotes/index.html",
               "name":"https://pillow.readthedocs.io/en/stable/releasenotes/index.html",
               "refsource":"MISC",
               "tags":[
                  "Release Notes",
                  "Vendor Advisory"
               ]
            },
            {
               "url":"https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html#buffer-overflow",
               "name":"https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html#buffer-overflow",
               "refsource":"MISC",
               "tags":[
                  "Release Notes",
                  "Vendor Advisory"
               ]
            },
            {
               "url":"https://lists.debian.org/debian-lts-announce/2021/07/msg00018.html",
               "name":"[debian-lts-announce] 20210722 [SECURITY] [DLA 2716-1] pillow security update",
               "refsource":"MLIST",
               "tags":[
                  "Mailing List",
                  "Third Party Advisory"
               ]
            },
            {
               "url":"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/7V6LCG525ARIX6LX5QRYNAWVDD2MD2SV/",
               "name":"FEDORA-2021-bf01a738f3",
               "refsource":"FEDORA",
               "tags":[
                  "Mailing List",
                  "Third Party Advisory"
               ]
            },
            {
               "url":"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/VUGBBT63VL7G4JNOEIPDJIOC34ZFBKNJ/",
               "name":"FEDORA-2021-3ec845dc0c",
               "refsource":"FEDORA",
               "tags":[
                  "Mailing List",
                  "Third Party Advisory"
               ]
            },
            {
               "url":"https://security.gentoo.org/glsa/202211-10",
               "name":"GLSA-202211-10",
               "refsource":"GENTOO",
               "tags":[

               ]
            }
         ]
      },
      "description":{
         "description_data":[
            {
               "lang":"en",
               "value":"Pillow through 8.2.0 and PIL (aka Python Imaging Library) through 1.1.7 allow an attacker to pass controlled parameters directly into a convert function to trigger a buffer overflow in Convert.c."
            }
         ]
      }
   },
   "configurations":{
      "CVE_data_version":"4.0",
      "nodes":[
         {
            "operator":"OR",
            "children":[

            ],
            "cpe_match":[
               {
                  "vulnerable":true,
                  "cpe23Uri":"cpe:2.3:a:python:pillow:*:*:*:*:*:*:*:*",
                  "versionStartIncluding":"1.2",
                  "versionEndIncluding":"8.2.0",
                  "cpe_name":[

                  ]
               },
               {
                  "vulnerable":true,
                  "cpe23Uri":"cpe:2.3:a:python:pillow:*:*:*:*:*:*:*:*",
                  "versionStartIncluding":"1.0",
                  "versionEndIncluding":"1.1.7",
                  "cpe_name":[

                  ]
               }
            ]
         },
         {
            "operator":"OR",
            "children":[

            ],
            "cpe_match":[
               {
                  "vulnerable":true,
                  "cpe23Uri":"cpe:2.3:o:debian:debian_linux:9.0:*:*:*:*:*:*:*",
                  "cpe_name":[

                  ]
               }
            ]
         },
         {
            "operator":"OR",
            "children":[

            ],
            "cpe_match":[
               {
                  "vulnerable":true,
                  "cpe23Uri":"cpe:2.3:o:fedoraproject:fedora:33:*:*:*:*:*:*:*",
                  "cpe_name":[

                  ]
               },
               {
                  "vulnerable":true,
                  "cpe23Uri":"cpe:2.3:o:fedoraproject:fedora:34:*:*:*:*:*:*:*",
                  "cpe_name":[

                  ]
               }
            ]
         }
      ]
   },
   "impact":{
      "baseMetricV3":{
         "cvssV3":{
            "version":"3.1",
            "vectorString":"CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H",
            "attackVector":"NETWORK",
            "attackComplexity":"LOW",
            "privilegesRequired":"NONE",
            "userInteraction":"NONE",
            "scope":"UNCHANGED",
            "confidentialityImpact":"HIGH",
            "integrityImpact":"HIGH",
            "availabilityImpact":"HIGH",
            "baseScore":9.8,
            "baseSeverity":"CRITICAL"
         },
         "exploitabilityScore":3.9,
         "impactScore":5.9
      },
      "baseMetricV2":{
         "cvssV2":{
            "version":"2.0",
            "vectorString":"AV:N/AC:L/Au:N/C:P/I:P/A:P",
            "accessVector":"NETWORK",
            "accessComplexity":"LOW",
            "authentication":"NONE",
            "confidentialityImpact":"PARTIAL",
            "integrityImpact":"PARTIAL",
            "availabilityImpact":"PARTIAL",
            "baseScore":7.5
         },
         "severity":"HIGH",
         "exploitabilityScore":10.0,
         "impactScore":6.4,
         "acInsufInfo":false,
         "obtainAllPrivilege":false,
         "obtainUserPrivilege":false,
         "obtainOtherPrivilege":false,
         "userInteractionRequired":false
      }
   },
   "publishedDate":"2021-07-13T17:15Z",
   "lastModifiedDate":"2022-11-22T06:15Z"
}

Vulnerability update through OSV parsing - Severity is KO

The following resource is used when parsing OSV data from ecosystem Alpine:v3.15 : https://storage.googleapis.com/cve-osv-conversion/osv-output/CVE-2021-34552.json

{
   "id":"CVE-2021-34552",
   "details":"Pillow through 8.2.0 and PIL (aka Python Imaging Library) through 1.1.7 allow an attacker to pass controlled parameters directly into a convert function to trigger a buffer overflow in Convert.c.",
   "affected":[
      {
         "package":{
            "name":"py3-pillow",
            "ecosystem":"Alpine:v3.15",
            "purl":"pkg:alpine/py3-pillow"
         },
         "ranges":[
            {
               "type":"ECOSYSTEM",
               "events":[
                  {
                     "introduced":"0"
                  },
                  {
                     "fixed":"8.3.0-r0"
                  }
               ]
            }
         ]
      }
   ],
   "references":[
      {
         "type":"WEB",
         "url":"https://pillow.readthedocs.io/en/stable/releasenotes/index.html"
      },
      {
         "type":"ADVISORY",
         "url":"https://pillow.readthedocs.io/en/stable/releasenotes/index.html"
      },
      {
         "type":"WEB",
         "url":"https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html#buffer-overflow"
      },
      {
         "type":"ADVISORY",
         "url":"https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html#buffer-overflow"
      },
      {
         "type":"ARTICLE",
         "url":"https://lists.debian.org/debian-lts-announce/2021/07/msg00018.html"
      },
      {
         "type":"WEB",
         "url":"https://lists.debian.org/debian-lts-announce/2021/07/msg00018.html"
      },
      {
         "type":"ARTICLE",
         "url":"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/7V6LCG525ARIX6LX5QRYNAWVDD2MD2SV/"
      },
      {
         "type":"WEB",
         "url":"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/7V6LCG525ARIX6LX5QRYNAWVDD2MD2SV/"
      },
      {
         "type":"ARTICLE",
         "url":"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/VUGBBT63VL7G4JNOEIPDJIOC34ZFBKNJ/"
      },
      {
         "type":"WEB",
         "url":"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/VUGBBT63VL7G4JNOEIPDJIOC34ZFBKNJ/"
      },
      {
         "type":"ADVISORY",
         "url":"https://security.gentoo.org/glsa/202211-10"
      }
   ],
   "modified":"2022-11-22T06:15:00Z",
   "published":"2021-07-13T17:15:00Z"
}

You can see that this vulnerability item from OSV does not contain any severity data hence the removal of the existing vulnerability in DT database.

melba-lopez commented 1 year ago

thanks @syalioune for verifying.

Is there a reason why any 1 source of vulnerability data would override another? Shouldn't they be completely independent unless adding an alias?

Also, I'm sure you saw in the video that the vulns come up duplicated (even though the aliasing is the same). There's a disadvantage for the end users because they want to catch everything possible. However, if they turn everything on then they get tons of duplicates, which causes too much overhead.

I know its not there now, but I think it would be great if the project incorporates this logic to allow for all vulnerabilities to be discovered. I've seen an instance where NVD doesn't catch everything, but another source (like OSS Index) catches something extra.

syalioune commented 1 year ago

Is there a reason why any 1 source of vulnerability data would override another? Shouldn't they be completely independent unless adding an alias?

I guess i'd have to do with the fact that OSV is mainly an aggregator of vulnerability databases, some of which are already mirrored by DT like Github Advisories. However, that is just a guess. @nscuro @msymons @sahibamittal should know better 👋 So far, we have the current situation from what I can see in the code :

Vulnerability from	Can update vulnerability from
OSS Index	NVD
OSV	NVD, GHSA

Also, I'm sure you saw in the video that the vulns come up duplicated (even though the aliasing is the same). There's a disadvantage for the end users because they want to catch everything possible. However, if they turn everything on then they get tons of duplicates, which causes too much overhead.

Yes, there's already https://github.com/DependencyTrack/dependency-track/issues/1994 to log that enhancement

nscuro commented 1 year ago

Thanks for the analysis @syalioune!

OSS Index can only create a CVE vulnerability if it doesn't already exist in DT's local database yet. If it does exist, OSS Index does not modify it. Until we have a good solution to track different vulnerability properties from multiple sources, the data from the authoritative source should always take precedence (for CVE: NVD, for GHSA: GitHub).

It looks like for OSV there is currently no check of whether a given vulnerability was created by a different source, and if that source is still enabled. If both conditions are true, OSV should not update the vulnerability details.

melba-lopez commented 1 year ago

@nscuro @syalioune do you happen to know which release these are targeted for? Would love to help if you need a tester ;-)

syalioune commented 1 year ago

I can provide a fix by end of this week. I believe it could be available at the very last on 4.8.0

syalioune commented 1 year ago

@melba-lopez I've made the PR. You can directly test with docker image syalioune/dependency-track:fix-2293 instead of official apiserver image.

leec94 commented 1 year ago

hi @syalioune , i'm interested in testing out the fix with your docker image syalioune/dependency-track:fix-2293

i tried to run it with the following commands

docker pull syalioune/dependency-track:fix-2293

docker tag syalioune/dependency-track:fix-2293 dependencytrack/bundled:latest

docker volume create --name dependency-track

docker run -d -m 8192m -p 8080:8080 --name dependency-track -v dependency-track:/data dependencytrack/bundled

then checked localhost:8080, but instead of seeing the UI I only see this page:

any advice on how to run your docker image? thank you

valentijnscholten commented 1 year ago

@leec94 The docs will be helpful when trying to run DT: https://docs.dependencytrack.org/getting-started/deploy-docker/

leec94 commented 1 year ago

thanks. i did look at those in trying to run - i'm following "Quickstart (Manual Execution)" steps in my message above. i also tried changing the docker-compose to use the image but it said the version was invalid. which set of steps do you recommend for running the changes in image syalioune/dependency-track:fix-2293?

valentijnscholten commented 1 year ago

Probably something like

docker pull syalioune/dependency-track:fix-2293
docker tag syalioune/dependency-track:fix-2293 dependencytrack/apiserver:latest
curl -LO https://dependencytrack.org/docker-compose.yml
docker-compose up -d

Not sure if everything is 100% in sync between latest frontend and the fix-2293 images.

leec94 commented 1 year ago

I was able to run the changes successfully, thanks for the help! In running @melba-lopez 's sample SBOM, it does look like with the implemented fix that no vulnerabilities are being marked back to "unassigned" severity when OSV is turned on. In my tests, with screenshots below, there are critical vulnerabilities seen when OSV is off, and when it's turned back on the critical vulnerabilities are still there.

One other thing I did notice was that when I first imported the SBOM, Sonatype + NVD + Github + OSV was on, and the vulnerability results didn't have the critical vulnerabilities appear, the results looked like the results from @melba-lopez 's video after OSV was turned on. It was only after turning off OSV that the critical vulnerabilities appeared. Is this a known issue? The NVD data should still populate, but looks like it's being affected by OSV data too.

With pictures, hope this helps:

I imported SBOM with the vulnerability analyzers Sonatype + NVD + OSV + Github ON. You can see no critical vulnerabilities appear yet.
I turned off OSV, and clicked "Reanalyze", so vulnerability analyzers Sonatype + NVD + Github ON. Now there are critical vulnerabilities showing.
I turned OSV back on. No changes in vulnerabilities.

syalioune commented 1 year ago

I imported SBOM with the vulnerability analyzers Sonatype + NVD + OSV + Github ON. You can see no critical vulnerabilities appear yet.

It can be that your Github database mirroring was still ongoing and was uncomplete at the time you performed the first analysis. I just tested again on my local workstation and vulnerabilities like GHSA-pw3c-h7wp-cvhx are found right off the bat.

leec94 commented 1 year ago

that's a good point. i'll try it again and keep an eye on the logs to make sure the GH DB completes. thanks!

leec94 commented 1 year ago

After letting it load for longer, the Github vulnerabilities did show up, so everything is all set then. Thanks for the help!

nscuro commented 1 year ago

This appears to be resolved, thanks all for the collaboration and special thanks to @syalioune for fixing!

github-actions[bot] commented 1 year ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

DependencyTrack / dependency-track