search

oomichi / try-kubernetes

12 stars 5 forks source link

Add cleanup after e2e build #62

Closed oomichi closed 5 years ago

oomichi commented 5 years ago

下記のように e2e バイナリのビルドで失敗。

$ sudo /usr/local/go/bin/go  run hack/e2e.go -- --build
...
+++ [1113 00:07:38] Deleting docker image k8s.gcr.io/kube-controller-manager:v1.12.2
+++ [1113 00:07:40] Deleting docker image k8s.gcr.io/cloud-controller-manager:v1.12.2
write /dev/stdout: no space left on device
!!! [1113 00:08:52] Call tree:
!!! [1113 00:08:52]  1: /home/ubuntu/go/src/k8s.io/kubernetes/build/lib/release.sh:387 kube::release::build_hyperkube_image(...)
!!! [1113 00:08:52]  2: /home/ubuntu/go/src/k8s.io/kubernetes/build/lib/release.sh:221 kube::release::create_docker_images_for_server(...)
!!! [1113 00:08:52]  3: /home/ubuntu/go/src/k8s.io/kubernetes/build/lib/release.sh:227 kube::release::build_server_images(...)
!!! [1113 00:08:52]  4: /home/ubuntu/go/src/k8s.io/kubernetes/build/lib/release.sh:86 kube::release::package_server_tarballs(...)
!!! [1113 00:08:52]  5: build/release.sh:45 kube::release::package_tarballs(...)
!!! [1113 00:08:52] previous Docker build failed
!!! [1113 00:08:52] Call tree:
!!! [1113 00:08:52]  1: /home/ubuntu/go/src/k8s.io/kubernetes/build/lib/release.sh:221 kube::release::create_docker_images_for_server(...)
!!! [1113 00:08:52]  2: /home/ubuntu/go/src/k8s.io/kubernetes/build/lib/release.sh:227 kube::release::build_server_images(...)
!!! [1113 00:08:52]  3: /home/ubuntu/go/src/k8s.io/kubernetes/build/lib/release.sh:86 kube::release::package_server_tarballs(...)
!!! [1113 00:08:52]  4: build/release.sh:45 kube::release::package_tarballs(...)
!!! [1113 00:08:52] previous tarball phase failed
Makefile:426: recipe for target 'quick-release' failed
make: *** [quick-release] Error 1
make: Leaving directory '/home/ubuntu/go/src/k8s.io/kubernetes'
2018/11/13 00:08:52 process.go:155: Step 'make -C /home/ubuntu/go/src/k8s.io/kubernetes quick-release' finished in 16m50.968509879s
2018/11/13 00:08:52 main.go:312: Something went wrong: failed to acquire k8s binaries: error during make -C /home/ubuntu/go/src/k8s.io/kubernetes quick-release: exit status 2
2018/11/13 00:08:52 e2e.go:81: err: exit status 1
exit status 1
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.9G     0  3.9G   0% /dev
tmpfs           799M  8.7M  790M   2% /run
/dev/vda1        78G   77G  616M 100% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
tmpfs           799M     0  799M   0% /run/user/1000
$ sudo du --max-depth=1 -h
4.0K    ./lxd
108K    ./snapd
4.0K    ./urandom
4.0K    ./misc
4.0K    ./initscripts
388K    ./systemd
4.0K    ./plymouth
8.0K    ./vim
12K     ./ubuntu-fan
28K     ./pam
12K     ./apparmor
4.0K    ./git
12K     ./AccountsService
4.0K    ./dbus
68G     ./docker
20K     ./dhcp
4.0K    ./python
12K     ./nfs
oomichi commented 5 years ago

exited 状態の docker コンテナを削除 → 少し減ったが不十分

$ sudo docker rm 4b6328669172 44b1b604a75c 57ff99f7fb61 4ab3fab5b615 72fd123134c4 057acfa749e5 18bf827e408d 5fde45d824a4 03e3259c6cdd
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.9G     0  3.9G   0% /dev
tmpfs           799M  8.7M  790M   2% /run
/dev/vda1        78G   74G  4.5G  95% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
tmpfs           799M     0  799M   0% /run/user/1000

docker イメージ削除をする

$ sudo docker images
REPOSITORY                                       TAG                            IMAGE ID            CREATED             SIZE
k8s.gcr.io/hyperkube-amd64                       v1.12.2                        c3922cce4929        17 minutes ago      635 MB
kube-build                                       build-7e185f8cc7-5-v1.10.4-1   34d37c5fe897        30 minutes ago      2.49 GB
kube-build                                       build-0a50dc92ba-5-v1.10.3-1   ef74e0b24f37        4 days ago          2.58 GB
kube-build                                       build-b11fd9fdab-5-v1.10.3-1   8e79791939ea        4 weeks ago         2.58 GB
kube-build                                       build-60f7a63cf9-5-v1.10.3-1   3c1e27e9cc01        4 weeks ago         2.58 GB
<none>                                           <none>                         5a08b7526869        4 weeks ago         2.58 GB
<none>                                           <none>                         a85aa419b5a0        4 weeks ago         2.58 GB
busybox                                          latest                         59788edf1f3e        5 weeks ago         1.15 MB
kube-build                                       build-d0b97871be-5-v1.10.3-1   90eb64d25746        6 weeks ago         2.58 GB
<none>                                           <none>                         8c7235f4d03e        6 weeks ago         2.58 GB
<none>                                           <none>                         4f1399d81c0f        6 weeks ago         2.58 GB
<none>                                           <none>                         f5b28fe49512        6 weeks ago         2.58 GB
<none>                                           <none>                         180b2246d3af        6 weeks ago         2.58 GB
<none>                                           <none>                         9224f7e4c0d0        6 weeks ago         2.58 GB
<none>                                           <none>                         922413c23c1f        6 weeks ago         2.58 GB
<none>                                           <none>                         177d2591786b        6 weeks ago         2.58 GB
<none>                                           <none>                         8a9783d6d180        6 weeks ago         2.58 GB
<none>                                           <none>                         101cb6e59528        6 weeks ago         2.58 GB
<none>                                           <none>                         27c55cbe7afc        7 weeks ago         2.58 GB
<none>                                           <none>                         e583e9a0f155        7 weeks ago         2.58 GB
<none>                                           <none>                         e6ccb97e1f46        7 weeks ago         2.58 GB
<none>                                           <none>                         8c5b72f852aa        7 weeks ago         2.58 GB
<none>                                           <none>                         8a612db146cb        7 weeks ago         2.58 GB
<none>                                           <none>                         efb3f0794b60        7 weeks ago         2.58 GB
<none>                                           <none>                         2856179e2ddf        7 weeks ago         2.58 GB
...
$ sudo docker rmi ef74e0b24f37 8e79791939ea 3c1e27e9cc01

docker volumeがたまっていたのが原因。docker volume rm <volume-id> で全て削除した。

oomichi commented 5 years ago

kube-test は k/kubernetes/issues/74623 で対応中。