search

sylabs / singularity-cri

The Singularity implementation of the Kubernetes Container Runtime Interface
Apache License 2.0
114 stars 29 forks source link

Integration with K8s, "Modify Kubelet config" instructions do not work. #378

Open adamwoolhether opened 3 years ago

adamwoolhether commented 3 years ago

I've managed to solve the issue, but I'm not sure if the manual config was needed due to something on my end or if it's an issue with documentation.

What are the steps to reproduce this issue?

  1. $ cat > /etc/default/kubelet <<EOF KUBELET_EXTRA_ARGS=--container-runtime=remote \ --container-runtime-endpoint=unix:///var/run/singularity.sock \ --image-service-endpoint=unix:///var/run/singularity.sock EOF
  2. sudo systemctl restart kubelet

What happens?

K8's does not use the singular runtime interface.

What were you expecting to happen?

When applying the arch-extender pod, K8's should use the new runtime interface to pull the images from Image: cloud.sylabs.io/library/slurm/arch-scheduler. However this results in ImagePullBackOff.

Instead I had to edit /etc/kubernetes/kubelet.env and add the KUBELET_EXTRA_ARGS var to /etc/systemd/system/kubelet.service

Any logs, error output, comments, etc?

All pods container to run with default cri, as determined with crictl crictl pods WARN[0000] runtime connect using default endpoints: [unix:///var/run/dockershim.sock unix:///run/containerd/containerd.sock unix:///run/crio/crio.sock]. As the default settings are now deprecated, you should set the endpoint instead.

k logs sif-scheduler-extender Error from server (BadRequest): container "sif-scheduler-extender" in pod "sif-scheduler-extender" is waiting to start: trying and failing to pull image ...

Environment?

OS distribution and version: CentOS Linux release 7.9.2009 (Core)

go version: go1.11 linux/amd64

go env: GOARCH="amd64" GOBIN="" GOCACHE="/root/.cache/go-build" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOOS="linux" GOPATH="/root/go" GOPROXY="" GORACE="" GOROOT="/usr/local/go" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64" GCCGO="gccgo" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="/root/arch-scheduler/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build165916451=/tmp/go-build -gno-record-gcc-switches

Singularity-CRI version: 1.0.0-beta.5

Singularity version: 3.6.4-1.el7

Kubernetes version: 1.17.4