Open tekumara opened 2 years ago
$ kubectl -n kube-image logs builder-btlsg rshared-tmp
/tmp is a mountpoint
+ nsenter -m -p -t 1 -- env 'PATH=/usr/sbin:/usr/bin:/sbin:/bin:/bin/aux' sh -c 'mount --make-rshared $_DIR'
mount: /tmp: Invalid argument
$ kubectl -n kube-image logs builder-btlsg rshared-buildkit
/var/lib/buildkit is a mountpoint
+ nsenter -m -p -t 1 -- env 'PATH=/usr/sbin:/usr/bin:/sbin:/bin:/bin/aux' sh -c 'mount --make-rshared $_DIR'
mount: /var/lib/buildkit: Invalid argument
$ kubectl -n kube-image logs builder-btlsg rshared-containerd
/var/lib/rancher is a mountpoint
+ nsenter -m -p -t 1 -- env 'PATH=/usr/sbin:/usr/bin:/sbin:/bin:/bin/aux' sh -c 'mount --make-rshared $_DIR'
mount: /var/lib/rancher: Invalid argument
FYI I'm using k3s in k3d, and have the same issue on macos and ubuntu.
$ k3d --version
k3d version v5.3.0
k3s version v1.22.6-k3s1 (default)
I am also seeing this issue. I'm trying to create the simplest k3d/k3s cluster and perform a kim build on it. If I switch to kind it works. I can reproduce using the following:
> k3d cluster create all-in-one -p 1233:1233@loadbalancer -p 1234:1234@loadbalancer
...
> ./bin/kim builder install --endpoint-addr 127.0.0.1
...
INFO[0074] Waiting on builder daemon availability...
INFO[0081] Waiting on builder daemon availability...
Error: timeout waiting for builder to become available
> kubectl get pods -n kube-image
NAME READY STATUS RESTARTS AGE
builder-t6n7j 0/2 CreateContainerError 0 34m
If I look at the status of the failed pod, I see:
containerStatuses:
- image: rancher/kim:v0.1.0-beta.7
imageID: ""
lastState: {}
name: agent
ready: false
restartCount: 0
started: false
state:
waiting:
message: 'failed to generate container "90cbca9c4fc190572d3e0f8488aea95c26b0ef03310f2ac9f8431ca331359413"
spec: failed to generate spec: path "/var/lib/buildkit" is mounted on "/"
but it is not a shared mount'
reason: CreateContainerError
- image: docker.io/moby/buildkit:v0.8.3
imageID: ""
lastState: {}
name: buildkit
ready: false
restartCount: 0
started: false
state:
waiting:
message: 'failed to generate container "657848459e41fa6fa8f5d444deb509d370fa0f7163584e7a9353bf1380e213c3"
spec: failed to generate spec: path "/tmp" is mounted on "/" but it is not
a shared mount'
reason: CreateContainerError
My system information is as follows:
macOS: 12.3.1
kim: v0.1.0-beta.7 (e597b9564b47213734787b3e0c540a635b250bbf)
k3d: v5.4.3
k3s: v1.23.6-k3s1 (default)
oh hai @ashlineldridge! 👋
@tekumara Hey Oliver! I didn't even recognise your username!
I last tested this with k3d v4.4.3 without issue. While it is possible that something in the k3s image has changed I think it more likely that k3d 5.x has changed how /tmp
is setup for the k3s container(s). It is possible that I have cheated in making this work on my dev box by modifying / to be rshared.
Expand below to see more.
kubectl describe pods -n kube-image builder-btls
``` Name: builder-btlsg Namespace: kube-image Priority: 0 Node: k3d-kim-server-0/172.22.0.2 Start Time: Sun, 27 Mar 2022 10:49:58 +1100 Labels: app=kim app.kubernetes.io/component=builder app.kubernetes.io/managed-by=kim app.kubernetes.io/name=kim component=builder controller-revision-hash=7bb6779b98 pod-template-generation=1 Annotations: