[Bug]: Disk Usage Warning on staging.

[ivan-kocienski-gfsc]

Description

What is taking up so much space on staging?

• logs (?) (i don't think they are saved to disk)
• database dumps (?) do they stick around?
• something else

[r-ferrier]

logged into the server to check this. A folder containing lots of old docker images called overlay2 is the culprit:

root@placecal-staging:/var/lib/docker# du -sh *
92K buildkit
33M containers
16M image
92K network
14G overlay2
16K plugins
4.0K    runtimes
4.0K    swarm
4.0K    tmp
4.0K    trust
28K volumes

running docker image prune --all reclaimed 2.9GB

It sounds like there is a bit of a known leak with dokku where it creates a large number of volumes for images that are stored in this folder. For now, this is not a major problem, but I can see all the gunk building up in here so if this happens again we should have a look at manually pruning the overlay2 folder (also might be a good idea to look into this on production)

https://stackoverflow.com/questions/46672001/is-it-safe-to-clean-docker-overlay2

[katjam]

Yes! I have come across this overlay2 thing before.

[ivan-kocienski-gfsc]

We may be able to claim some of this back by removing metabase from staging/production.