search

vmware-archive / buildkit-cli-for-kubectl

BuildKit CLI for kubectl is a tool for building container images with your Kubernetes cluster
Other
499 stars 41 forks source link

creating a builder instance in k0s does not seem to work #134

Open rgl opened 2 years ago

rgl commented 2 years ago

What steps did you take and what happened

I'm trying this project with k0s v1.23.6+k0s.2, but it fails to mount various containerd directories: 

root@awx:~# kubectl-buildkit create \
>     --progress=plain \
>     --worker=containerd \
>     --containerd-sock=/var/run/k0s/containerd.sock \
>     --containerd-namespace=k8s.io
#1 [internal] booting buildkit
#1 waiting for 1 pods to be ready for buildkit
#1 0.040 Normal     buildkit-75bb679db9     SuccessfulCreate    Created pod: buildkit-75bb679db9-gswgp
#1 0.058 Normal     buildkit-75bb679db9-gswgp   Scheduled   Successfully assigned default/buildkit-75bb679db9-gswgp to awx
#1 0.058 Warning    buildkit-75bb679db9-gswgp   FailedMount     MountVolume.SetUp failed for volume "var-lib-containerd" : hostPath type check failed: /var/lib/containerd is not a directory
#1 0.058 Warning    buildkit-75bb679db9-gswgp   FailedMount     Unable to attach or mount volumes: unmounted volumes=[var-lib-containerd], unattached volumes=[var-lib-containerd run-containerd var-log tmp kube-api-access-hxll9 buildkitd-config containerd-sock var-lib-buildkit]: timed out waiting for the condition
#1 0.058 Warning    buildkit-75bb679db9-gswgp   FailedMount     Unable to attach or mount volumes: unmounted volumes=[var-lib-containerd], unattached volumes=[run-containerd var-log tmp kube-api-access-hxll9 buildkitd-config containerd-sock var-lib-buildkit var-lib-containerd]: timed out waiting for the condition

Any idea what is might be wrong? Or how to make it work?

FWIW, The usage/test of k0s was introduced in https://github.com/vmware-tanzu/buildkit-cli-for-kubectl/pull/44 by @adamparco and @dhiltgen merged the changes.

This might be because k0s launches containerd with non-default directories:

/var/lib/k0s/bin/containerd \
    --root=/var/lib/k0s/containerd \
    --state=/var/lib/k0s/run/containerd \
    --address=/var/lib/k0s/run/containerd.sock \
    --config=/etc/k0s/containerd.toml

And because there is only kubectl-buildkit arguments to set --containerd-sock=/var/run/k0s/containerd.sock?

What did you expect to happen

To use this project in a k0s k8s cluster.

Environment Details:

Builder Logs

There are no logs because the builder was not even started:

root@awx:~# kubectl logs -l app=buildkit
No resources found in default namespace.

Vote on this request

This is an invitation to the community to vote on issues. Use the "smiley face" up to the right of this comment to vote.