adoptium / infrastructure

This repo contains all information about machine maintenance.
Apache License 2.0
85 stars 101 forks source link

System unavailable: test-docker-debian12-x64-1 - out of disk space #3680

Open zzambers opened 1 month ago

zzambers commented 1 month ago

System: test-docker-debian12-x64-1

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10666/console

System seems to be running out of disk space:

...
16:01:20  java.nio.file.FileSystemException: /home/jenkins/workspace/Grinder: No space left on device
16:01:20    at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
16:01:20    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
16:01:20    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
16:01:20    at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397)
16:01:20    at java.base/java.nio.file.Files.createDirectory(Files.java:700)
16:01:20    at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
16:01:20    at java.base/java.nio.file.Files.createDirectories(Files.java:793)
16:01:20    at hudson.FilePath.mkdirs(FilePath.java:3751)
16:01:20    at hudson.FilePath$Mkdirs.invoke(FilePath.java:1416)
16:01:20    at hudson.FilePath$Mkdirs.invoke(FilePath.java:1411)
16:01:20    at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3613)
16:01:20    at hudson.remoting.UserRequest.perform(UserRequest.java:211)
16:01:20    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
16:01:20    at hudson.remoting.Request$2.run(Request.java:377)
16:01:20    at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
16:01:20    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
16:01:20    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
16:01:20    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
16:01:20    at java.base/java.lang.Thread.run(Thread.java:840)
...
sxa commented 1 month ago

Noting that this is on the skytap x64 host which I brought online last week to experiment a bit with the Solaris VM that was on there ... I'll mark that container offline for now ...

sxa commented 1 month ago

Both containers on the host marked offline to new jobs as both will be affected:

FYI @judovana as you have Grinders running on both of these just now which may well suffer the same fate (10671 and 10672)

judovana commented 1 month ago

yah I noted, and killed them already :( TY!