DependencyTrack / dependency-track

Dependency-Track is an intelligent Component Analysis platform that allows organizations to identify and reduce risk in the software supply chain.
https://dependencytrack.org/
Apache License 2.0
2.58k stars 542 forks source link

Disk space used #320

Open h3xstream opened 5 years ago

h3xstream commented 5 years ago

Issue Type:

Current Behavior:

Docker volume keeps growing.. I suspect that this is cause by the recurring downloads.

> docker system df -v | grep dependency
owasp/dependency-track                              latest              cb34a56eebbc        3 months ago        168.8MB             4.413MB             164.4MB             1
acb1ef891df1        owasp/dependency-track                            "/bin/sh -c 'java $J…"   1                   1GB                 10 days ago         Exited (255) 26 seconds ago   dependency-track_server_1

Steps to Reproduce (if defect):

version: '2'
services:
  server:
    image: owasp/dependency-track
    ports:
     - "8080:8080"

1.

docker-compose up
  1. wait for the download to complete..
  2. Ctrl-C
  3. docker-compose up
  4. After each restart, check the volume sizes.
    > docker system df -v | grep dependency

NPM, NVD-CVE seems to be redownload on every restart.

Expected Behavior:

Download incrementally.

Environment:

stevespringett commented 5 years ago

Content from NVD, NPM, and (if enabled) VulnDB, will be synced upon server start-up, and every 24 hours after that.

There should be a data volume for the container as well. Once the contents of the NVD are stored on the volume, redownloads of the NVD will only occur for the years (2002 - present) for which there are changes. If there are no changes to a years feed, it will not be downloaded again on startup or daily until there is such a change. NPM will always be synchronized but is a much faster process without as much overhead.

https://docs.dependencytrack.org/getting-started/deploy-docker/

h3xstream commented 5 years ago

Thanks. I will try to restart from scratch and use mount volume.

Drewster727 commented 5 years ago

I have a similar but slightly different issue with docker and dependency-track. I noticed my disk usage kept growing and growing. Narrowing it down to the container files:

[root@server-01 tmp]# pwd
/data/docker/overlay/8631c335d25a4e17f27f35499f50d4820401c6e09162f03dd255ece9b3f3fadf/merged/tmp

The overlay (container) files grow steadily over time in the /tmp folder: image

These folders created by dep-track are 132MB each and keep piling up: image

I'm not doing anything out of the ordinary with my container start: docker run -d -v /data/container_volumes/dependency-track:/data -p 8080:8080 owasp/dependency-track

This very well could be caused with my continuous container restarts every few hours. I did that due to instability. Can I shut this off or have the logs rotate?

Thanks, Drew

stevespringett commented 5 years ago

@Drewster727 if you've upgraded to 3.4.1, you can safely remove the continuous restarts. They are no longer necessary.

Logs are rotated by default. 10MB per log and up to 9 saved. These are defined here: https://github.com/DependencyTrack/dependency-track/blob/3.4.x/src/main/docker/logback.xml

Drewster727 commented 5 years ago

thanks @stevespringett

Najafov007 commented 2 weeks ago

Content from NVD, NPM, and (if enabled) VulnDB, will be synced upon server start-up, and every 24 hours after that.

There should be a data volume for the container as well. Once the contents of the NVD are stored on the volume, redownloads of the NVD will only occur for the years (2002 - present) for which there are changes. If there are no changes to a years feed, it will not be downloaded again on startup or daily until there is such a change. NPM will always be synchronized but is a much faster process without as much overhead.

https://docs.dependencytrack.org/getting-started/deploy-docker/

Hello! So recently We've got the same issue, we see that our Openshift reports us that volume has been dedicated to apiserver ( 5GB) ran out. So we wanted to know, how much approximately will the whole data from 2002 - present be? We also plan to connect a DB, but problem is our pod is not running right now Thank you!