DependencyTrack / dependency-track

Dependency-Track is an intelligent Component Analysis platform that allows organizations to identify and reduce risk in the software supply chain.
https://dependencytrack.org/
Apache License 2.0
2.58k stars 542 forks source link

Increasing memory usage during the PortfolioMetricsUpdateTask #4127

Open mzweem opened 2 weeks ago

mzweem commented 2 weeks ago

Current Behavior

Hi, we currently run Dependency Track with 1200 projects, with around 160k components and we notice a certain build up of memory usage during the day: It usally starts with around 1GB in use, but steadily builds up to 10GB.

image

I see in the logging that the PortfolioMetricsUpdateTask finishes when the memory usage drops down to 1GB. Is this normal behaviour? I have set the Portfolio Metrics update interval set to 1 hour. This is exactly the time after the memory drop before it starts building up again.

Steps to Reproduce

  1. Run Dependeny Track 4.11.7 with the official containers.
  2. Let it run with 1200 projects, 160k components. Portfolio metrics interval set to 1 hour.
  3. The task starts, and keeps using more memory.
  4. Task finishes and all the memory is released again.

Expected Behavior

Perhaps a bit more leveled out memory usage?

Dependency-Track Version

4.11.7

Dependency-Track Distribution

Container Image

Database Server

PostgreSQL

Database Server Version

16

Browser

N/A

Checklist

nscuro commented 5 days ago

The API server is allowed to claim up to 90% of available memory for the JVM's heap per default: https://github.com/DependencyTrack/dependency-track/blob/b4edb30fcee9c75019391bb305bdd0534e087ea5/src/main/docker/Dockerfile#L19

Until heap usage approaches the maximum allowed size, there's little incentive for the garbage collector to kick in. The steep fall after heap usage reaches the ~10G mark in your case shows this quite well. Unless the application crashes, this is nothing to be worried about.

Also note that you're looking at specifically JVM heap usage there. The actual memory used by the JVM doesn't fluctuate like this, and in your case likely sits around the 11-12G mark.

If you don't want the heap to grow that large, you can set the EXTRA_JAVA_OPTIONS environment variable to either: