search

DependencyTrack / dependency-track

Dependency-Track is an intelligent Component Analysis platform that allows organizations to identify and reduce risk in the software supply chain.
https://dependencytrack.org/
Apache License 2.0
2.6k stars 552 forks source link

Metrics update for project - FAILS #1957

Closed Andersrapp closed 2 years ago

Andersrapp commented 2 years ago

Current Behavior:

A version 1557 of a project is uploaded to our internal DependencyTrack on August 19 2022. It is a maven project with snakeyaml:1.29 which is included in the uploaded bom. It is not flagged as a vulnerability since the vulnerability was only published on August 30.

Another version 1563 of the same project is uploaded to our internal DependencyTrack on September 8 with the same dependency snakeyaml:1.29 which is included in the bom. It is flagged as a vulnerability --> CVE-2022-25857.

We turn on DEBUG logging and check logs for the project uuid for the first version 1557. It prints the following:

Indexing 160 components in project: xxxx Completed indexing of 160 components in project: xxxx Executing metrics update for project: xxxx Retrieving all components for project: xxxx Retrieving existing audited count for project: xxxx Retrieving existing suppression count for project: xxxx Metrics are unchanged for project: xxxx Persisting metrics for project: xxxx Completed metrics update for project: xxxx

Looking at the first uploaded version (1557) of the project and the list of components we find snakeyaml:1.29 listed with 0 vulnerabilities and a "Risk Score" of 0.

Looking at the second uploaded version (1563) of the project and the list of components we find snakeyaml:1.29 listed with 1 vulnerabilities (marked orange|yellow) and a "Risk score" of 8.

Looking at the list of vulnerabilities we find snakeyaml:1.29 listed with a number of "affected projects". Looking at those projects we find only the second uploaded version (1563) of the project.

There are more versions uploaded but for the sake of this discussion only versions uploaded after the vulnerability is published are listed as "affected projects" under vulnerabilities or has the snakeyaml marked as a vulnerability in the projects list of "Components".

This behavior also include other dependencies and vulnerabilities like io.undertow:undertow-core:2.2.18.Final with CVE-2022-2764 which was published on September 1. It shows up as a vulnerability on project versions uploaded after but not on project versions uploaded before vulnerability publication date. Checking the logs we see that the re-scanning, just like in the previous example, considers "Metrics are unchanged for project" for the previously uploaded project versions even though both older and later project versions contain the same dependency and version.

Steps to Reproduce:

Not sure but one way could be to:

  1. Start a server with old versions of the NVD Database / OSS Index, etc.
  2. Upload a project with either the snakeyaml or undertow-core vulnerability
  3. Update all databases
  4. Upload a new version of the same project.
  5. Check whether the vulnerabilities list has both project versions in list of Components or not
  6. Check whether both project versions list the vulnerable dependency version with the same Vulnerability and Risk Score or not

Expected Behavior:

  1. All versions of the project (with the relevant vulnerability) are listed with the same Vulnerability and Risk Score in the projects list of Components after re-scanning.
  2. All versions of the project (with the relevant vulnerability) are listed in the Vulnerabilities "Affected Projects" after re-scanning.

Logically the vulnerability should point to the same object? Why the same dependency version is evaluated differently in different places is inexplicable?

In the documentation on https://docs.dependencytrack.org/usage/cicd/ it states: "Dependency-Track continuously monitors components for known vulnerabilities....All components in Dependency-Track, regardless of changes, are automatically analyzed on a daily basis."

We see that the re-scanning takes place every hour but it's incapable of producing any new relevant result which makes it completely pointless.

Environment:

Docker-compose version 3.7 ​ ​ ​- Dependency-Track Version: 4.5.0 ​- Distribution: Docker ​- BOM Format & Version: CycloneDX:1.4 ​- Database Server: PostgreSQL ​- Browser: Any

dependency_track_3 dependency_track_2 dependency_track_1

nscuro commented 2 years ago

Indeed DT should identify new vulnerabilities in older projects.

However, the metrics update that runs hourly does not perform vulnerability analysis, it just calculates metrics based on the information that's in the database already. Vulnerability analysis, both ad-hoc when uploading a BOM and continuously for the entire portfolio, is performed by the VulnerabilityAnalysisTask.

What you want to look out for in the logs is messages along the lines of:

Analyzing <NUMBER> components in project: <PROJECT_UUID>

Completed analysis of <NUMBER> components in project: <PROJECT_UUID>

A scan of the entire portfolio is kicked off every 24 hours per default. This interval will be configurable in DT 4.6. A portfolio scan also explicitly excludes projects that have been marked as "inactive".

In your case, it seems like the regular VulnerabilityAnalysisTask execution is not happening, is running into errors, or the project in question has been marked as inactive.

Please check the API server logs for the pattern mentioned above.

For reference, here is where these log messages are generated:

https://github.com/DependencyTrack/dependency-track/blob/a3b42a6fe4d19fc91122887dba0841f26fa7c7c4/src/main/java/org/dependencytrack/tasks/VulnerabilityAnalysisTask.java#L88-L91

Andersrapp commented 2 years ago

Thanks for your reply. I need to update some things. We discovered some errors on our side. The container was restarting on a regular basis since we were running the docker-compose with less memory (12GB) than recommended (16GB I believe). We now run the service on a server with 4 CPUs and a total 32 GB RAM. Docker-compose is set to

limitations: 24GB required: 16GB

Looking at the containers now they've been running for three days total so the issue with the container restarting (and likely failing to run certain tasks) should be fixed. The number of identified projects with the snakeyaml vulnerability has more than doubled and we can see a lot of repositories that were uploaded before publication now lists the vulnerability correctly. It does not work on all projects though. There are several projects that are "ignored" somehow.

Looking at the logs for VulnerabilityAnalysisTask they contain very little information. It says "Analyzing" and "Completed" and mentions the number of components in the project. However, it does not reveal anything on additional vulnerabilities discovered. This could be quite helpful.

Not sure if the re-scan runs on all projects or just active ones but just to clarify: all the projects we're looking at are set as Active.

I just grepped the logs on a project id we're wondering about and all that came back was RepositoryMetaAnalyzerTask logs. I'll keep digging the logs and get back to you but any input from you is helpful.

Andersrapp commented 2 years ago

Can you answer one question? Does the re-scan run on all projects or only on active ones?

nscuro commented 2 years ago

As per my previous comment:

A portfolio scan also explicitly excludes projects that have been marked as "inactive".

Andersrapp commented 2 years ago

Thanks! You can close this. There's still an issue with a few projects getting ignored in the scan. Unfortunately, we have few examples and too little logs so we've run out of leads.

nscuro commented 2 years ago

Ok, please feel free to open another issue if the issue materialized into something reproducible.

FYI, this is where the to-be-scanned projects are queried:

https://github.com/DependencyTrack/dependency-track/blob/bcc57feb31e44295344d2d59d28a79c951dfd842/src/main/java/org/dependencytrack/tasks/VulnerabilityAnalysisTask.java#L76-L93

qm.getAllProjects(true) fetches all active projects. Beside that, there's no filtering going on.

github-actions[bot] commented 1 year ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.