Setting Up Pumba on Kind Kubernetes Cluster

[jujoramos]

Hello, I'm trying to use pumba within a local kind Kubernetes cluster on macOs. For some reason I can't understand yet, the tool itself is not able to find any containers (and the logs, even when set to debug, don't show much).

Steps to Reproduce

1. Create a local kind cluster using the following script (taken from https://github.com/tilt-dev/kind-local/blob/master/kind-with-registry.sh and modified to to allow access to the docker daemon socket):

   
   #!/bin/sh
   #
   # This script came from https://github.com/tilt-dev/kind-local/blob/master/kind-with-registry.sh
   # See https://github.com/tilt-dev/kind-local
   #
   # Adapted from:
   # https://github.com/kubernetes-sigs/kind/commits/master/site/static/examples/kind-with-registry.sh
   #
   # Copyright 2020 The Kubernetes Project
   #
   # Licensed under the Apache License, Version 2.0 (the "License");
   # you may not use this file except in compliance with the License.
   # You may obtain a copy of the License at
   #
   #     http://www.apache.org/licenses/LICENSE-2.0
   #
   # Unless required by applicable law or agreed to in writing, software
   # distributed under the License is distributed on an "AS IS" BASIS,
   # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   # See the License for the specific language governing permissions and
   # limitations under the License.

set -o errexit

desired cluster name; default is "kind"

KIND_CLUSTER_NAME="${KIND_CLUSTER_NAME:-kind}"

kind_version=$(kind version) kind_network='kind' reg_name='kind-registry' reg_port='5000' case "${kind_version}" in "kind v0.7." | "kind v0.6." | "kind v0.5."*) kind_network='bridge' ;; esac

create registry container unless it already exists

running="$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)" if [ "${running}" != 'true' ]; then docker run \ -d --restart=always -p "${reg_port}:5000" --name "${reg_name}" \ registry:2 fi

reg_host="${reg_name}" if [ "${kind_network}" = "bridge" ]; then reg_host="$(docker inspect -f '{{.NetworkSettings.IPAddress}}' "${reg_name}")" fi echo "Registry Host: ${reg_host}"

create a cluster with the local registry enabled in containerd

cat <<EOF | kind create cluster --name "${KIND_CLUSTER_NAME}" --config=- kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes:

• role: control-plane extraMounts:
  • hostPath: /var/run/docker.sock containerPath: /var/run/docker.sock containerdConfigPatches:
• |- [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:${reg_port}"] endpoint = ["http://${reg_host}:${reg_port}"] EOF

cat <<EOF | kubectl apply -f - apiVersion: v1 kind: ConfigMap metadata: name: local-registry-hosting namespace: kube-public data: localRegistryHosting.v1: | host: "localhost:${reg_port}" help: "https://kind.sigs.k8s.io/docs/user/local-registry/" EOF

if [ "${kind_network}" != "bridge" ]; then containers=$(docker network inspect ${kind_network} -f "{{range .Containers}}{{.Name}} {{end}}") needs_connect="true" for c in $containers; do if [ "$c" = "${reg_name}" ]; then needs_connect="false" fi done if [ "${needs_connect}" = "true" ]; then
docker network connect "${kind_network}" "${reg_name}" || true fi fi

2. Deployment for testing purposes:

cat <<EOF | kubectl apply -f - apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers:

• name: nginx image: nginx:1.14.2 ports:
  • containerPort: 80 EOF

    3. Deploy `pumba` to kill the containers every 30 seconds:

    cat <<EOF | kubectl apply -f - apiVersion: apps/v1 kind: DaemonSet metadata: name: pumba spec: selector: matchLabels: app: pumba template: metadata: labels: app: pumba com.gaiaadm.pumba: "true" name: pumba spec: containers:

• image: gaiaadm/pumba imagePullPolicy: Always name: pumba-kill args:
  • --random
  • --log-level
  • debug
  • --label
  • app=nginx
  • --interval
  • 30s
  • kill securityContext: capabilities: add: ["NET_ADMIN"] resources: requests: cpu: 10m memory: 5M limits: cpu: 100m memory: 20M volumeMounts:
  • name: dockersocket mountPath: /var/run/docker.sock volumes:
• hostPath: path: /var/run/docker.sock name: dockersocket EOF

The logs for the container show the following:

│ time="2020-10-05T10:33:17Z" level=debug msg="killing all matching containers"                                                                                                                                                              │
│ time="2020-10-05T10:33:17Z" level=debug msg="listing matching containers" labels="[app=nginx]" limit=0 names="[]" pattern= random=true                                                                                                     │
│ time="2020-10-05T10:33:17Z" level=debug msg="listing containers"                                                                                                                                                                           │
│ time="2020-10-05T10:33:17Z" level=warning msg="no containers to kill"                                                                                                                                                                      │
│ time="2020-10-05T10:33:47Z" level=debug msg="next chaos execution (tick) ..."                                                                                                                                                              │
│ time="2020-10-05T10:33:47Z" level=debug msg="killing all matching containers"                                                                                                                                                              │
│ time="2020-10-05T10:33:47Z" level=debug msg="listing matching containers" labels="[app=nginx]" limit=0 names="[]" pattern= random=true                                                                                                     │
│ time="2020-10-05T10:33:47Z" level=debug msg="listing containers"                                                                                                                                                                           │
│ time="2020-10-05T10:33:47Z" level=warning msg="no containers to kill"  

FYI, I've tried steps 2 - 3 directly on a Kubernetes cluster created on GKE and it works just fine, I can see the pods correctly being killed. Still, it would be great to make it work correctly on a local Kubernetes cluster as it greatly reduces the time needed to test and develop. Thanks in advance!.

[jujoramos]

Okay, looks like kind uses containerd and not docker as the underlying container engine, that's why it's not working... will try with minikube instead. Closing this issue now.