test-scaleway-ubuntu1604-x64-1 keeps running out of disc space

adoptium / infrastructure

This repo contains all information about machine maintenance.

Apache License 2.0

86 stars 102 forks source link

test-scaleway-ubuntu1604-x64-1 keeps running out of disc space #528

Closed Mesbah-Alam closed 6 years ago

Mesbah-Alam commented 6 years ago

test-scaleway-ubuntu1604-x64-1 seems to keep running out of disc space.

2:40:25 [ssh-agent]   Java/JNR ssh-agent
12:40:25 ERROR: [ssh-agent] Could not register BouncyCastle on the remote agent.
12:40:25 java.nio.file.FileSystemException: /tmp/resource-2065839481020573047: No space left on device

https://ci.adoptopenjdk.net/view/work%20in%20progress/job/Grinder/281/console

Mesbah-Alam commented 6 years ago

FYI.. @sxa555

Mesbah-Alam commented 6 years ago

More build failure due to this issue: https://ci.adoptopenjdk.net/view/Test_external/job/openjdk11_hs_externaltest_x86-64_linux/9/consoleFull

karianna commented 6 years ago

I've cleaned up the /tmp directory but the / partition is still almost full. Running ncdu to see what it is.

karianna commented 6 years ago

OK it's /var/lib/docker/overlay2 that takes up 35GB or so. Is this Docker image/container related? Is some sort of cleanup not occurring?

Mesbah-Alam commented 6 years ago

overlay2 is not related to any test docker container according to my knowledge.

karianna commented 6 years ago

OK, so I had to clear all images/containers and volumes - which cleaned out that dir. There's plenty of space left now but I think our docker based jobs need to clean up after themselves better.