search

DependencyTrack / dependency-track

Dependency-Track is an intelligent Component Analysis platform that allows organizations to identify and reduce risk in the software supply chain.
https://dependencytrack.org/
Apache License 2.0
2.66k stars 570 forks source link

dependency-track high cpu usage #1370

Open mawl opened 2 years ago

mawl commented 2 years ago

Current Behavior:

image

image

Expected Behavior:

No high cpu usage, no cascading effect.

Environment:

Additional Details:

Looks like https://github.com/DependencyTrack/dependency-track/issues/264

This is found in the logs during every cpu usage increasement:

2022-02-03 04:24:28,504 [] INFO [org.dependencytrack.tasks.MetricsUpdateTask] Executing metrics update on vulnerability database 2022-02-03 04:24:39,924 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Internal analysis complete 2022-02-03 04:24:39,929 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:25:04,837 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Node Audit analysis complete 2022-02-03 04:25:04,843 [] WARN [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access 2022-02-03 04:25:04,843 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task 2022-02-03 04:25:16,616 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Analyzing 2 component(s) 2022-02-03 04:25:16,616 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2022-02-03 04:25:16,662 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Completed analysis of 2 components in project: c5caee7c-a6c2-4b2c-b3fb-ec6d2a5908f9 2022-02-03 04:25:16,664 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Analyzing 2 components in project: 6ea8315e-056d-4975-bea8-a597fcca12c9 2022-02-03 04:25:16,684 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Starting internal analysis task 2022-02-03 04:25:26,515 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2022-02-03 04:25:26,548 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Completed analysis of 180 components in project: be7c886d-1e7e-4fb5-abf1-a8bd53b18533 2022-02-03 04:25:26,552 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Analyzing 180 components in project: 960c8763-996e-4c24-a9c3-f94c38ef8c31 2022-02-03 04:25:26,569 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Starting internal analysis task 2022-02-03 04:25:30,417 [] ERROR [org.dependencytrack.tasks.VulnerabilityAnalysisTask] An unexpected error occurred performing a vulnerability analysis task 2022-02-03 04:25:30,436 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:25:40,422 [] ERROR [org.dependencytrack.tasks.VulnerabilityAnalysisTask] An unexpected error occurred performing a vulnerability analysis task 2022-02-03 04:25:40,439 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:25:41,937 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2022-02-03 04:25:41,945 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Completed analysis of 3 components in project: 447bee2e-4897-4599-a917-e0010eed315f 2022-02-03 04:25:41,948 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Analyzing 126 components in project: 2922eb84-ed0f-4316-aed7-7585fb0e3c70 2022-02-03 04:25:41,954 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Starting internal analysis task 2022-02-03 04:25:57,754 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Internal analysis complete 2022-02-03 04:25:57,759 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:26:18,917 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Node Audit analysis complete 2022-02-03 04:26:18,926 [] WARN [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access 2022-02-03 04:26:18,926 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task 2022-02-03 04:26:41,163 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Node Audit analysis complete 2022-02-03 04:26:41,183 [] WARN [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access 2022-02-03 04:26:41,183 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task 2022-02-03 04:26:44,433 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Node Audit analysis complete 2022-02-03 04:26:44,451 [] WARN [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access 2022-02-03 04:26:44,451 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task 2022-02-03 04:26:52,197 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2022-02-03 04:26:52,206 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Completed analysis of 126 components in project: 2922eb84-ed0f-4316-aed7-7585fb0e3c70 2022-02-03 04:26:52,210 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Analyzing 175 components in project: 8e0ac3d5-c153-4f13-b6a9-4359a19f1baf 2022-02-03 04:26:52,215 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Starting internal analysis task 2022-02-03 04:27:06,400 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Internal analysis complete 2022-02-03 04:27:06,407 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:27:19,571 [] INFO [org.dependencytrack.tasks.InternalComponentIdentificationTask] Internal component identification task completed 2022-02-03 04:27:29,689 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Node Audit analysis complete 2022-02-03 04:27:29,695 [] WARN [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access 2022-02-03 04:27:29,695 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task 2022-02-03 04:28:03,175 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2022-02-03 04:28:03,184 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Completed analysis of 175 components in project: 8e0ac3d5-c153-4f13-b6a9-4359a19f1baf 2022-02-03 04:28:03,196 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Analyzing 478 components in project: 6ae04e1f-6758-4d8a-8a7c-7ef25309656e 2022-02-03 04:28:03,207 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Starting internal analysis task 2022-02-03 04:28:04,517 [] INFO [org.dependencytrack.tasks.MetricsUpdateTask] Completed metrics update on vulnerability database 2022-02-03 04:28:14,383 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Internal analysis complete 2022-02-03 04:28:14,389 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:28:35,545 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Node Audit analysis complete 2022-02-03 04:28:35,550 [] WARN [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access 2022-02-03 04:28:35,550 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task 2022-02-03 04:28:40,516 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2022-02-03 04:28:40,549 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Completed analysis of 180 components in project: 960c8763-996e-4c24-a9c3-f94c38ef8c31 2022-02-03 04:28:40,573 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Analyzing 1641 components in project: 9e5c6eda-39b4-433e-b8e0-a3ea36ce0f7a 2022-02-03 04:28:40,606 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Starting internal analysis task 2022-02-03 04:28:46,629 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2022-02-03 04:28:46,664 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Completed analysis of 2 components in project: 6ea8315e-056d-4975-bea8-a597fcca12c9 2022-02-03 04:28:46,666 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Analyzing 2 components in project: cd225ca1-608b-4e4c-a801-095069e12f53 2022-02-03 04:28:46,733 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Starting internal analysis task 2022-02-03 04:28:52,653 [] ERROR [org.dependencytrack.tasks.VulnerabilityAnalysisTask] An unexpected error occurred performing a vulnerability analysis task 2022-02-03 04:28:52,670 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:28:58,877 [] ERROR [org.dependencytrack.tasks.VulnerabilityAnalysisTask] An unexpected error occurred performing a vulnerability analysis task 2022-02-03 04:28:58,896 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:29:10,112 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Sonatype OSS Index analysis complete 2022-02-03 04:29:10,121 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Completed analysis of 478 components in project: 6ae04e1f-6758-4d8a-8a7c-7ef25309656e 2022-02-03 04:29:10,129 [] INFO [org.dependencytrack.tasks.VulnerabilityAnalysisTask] Analyzing 478 components in project: 00ae7a90-8dd0-4be1-a2b6-ab1d2b0c8acc 2022-02-03 04:29:10,139 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Starting internal analysis task 2022-02-03 04:29:20,817 [] INFO [org.dependencytrack.tasks.scanners.InternalAnalysisTask] Internal analysis complete 2022-02-03 04:29:20,823 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Starting Node Audit analysis task 2022-02-03 04:29:41,604 [] INFO [org.dependencytrack.tasks.scanners.NpmAuditAnalysisTask] Node Audit analysis complete 2022-02-03 04:29:41,610 [] WARN [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] An API username or token has not been specified for use with OSS Index. Using anonymous access 2022-02-03 04:29:41,610 [] INFO [org.dependencytrack.tasks.scanners.OssIndexAnalysisTask] Starting Sonatype OSS Index analysis task

Thanks for your help.

ruckc commented 2 years ago

I believe this is due to the large (1500) number of projects. Every hour (i think) dependency-track runs analysis, which is essentially a sequential forloop that iterates over each project to update the dashboard metrics/charts. I was load testing DT with 3000 projects, and this task essentially ran 24/7.

mawl commented 2 years ago

Seems to happen every six hours:

image

As a workaround we restart the apiserver container every day so that cpu usage drops again.

stevespringett commented 2 years ago

Duplicate of #1212

alenloncaric commented 9 months ago

Hi Gents,

using the latest 4.10.1 dependency track boundle docker and the CPU is always 100% on one core. image

Version info: image

Any idea what should i disable to get of this cpu usage?

With kind regards, Alen