windmill-labs / windmill

Open-source developer platform to turn scripts into workflows and UIs. Fastest workflow engine (5x vs Airflow). Open-source alternative to Airplane and Retool.
https://windmill.dev
Other
9.59k stars 437 forks source link

feature: run docker scripts in a k8s cluster #2696

Open tdeckers-cisco opened 9 months ago

tdeckers-cisco commented 9 months ago

I've updated the docker 'language' script to be able to run container scripts in a kubernetes cluster. Not sure where/how to contribute this as an alternative to the docker remote execution.

# shellcheck shell=bash
# Bash script that calls docker as a client to the host daemon
# See documentation: https://www.windmill.dev/docs/advanced/docker
msg="${1:-world}"

IMAGE="docker/whalesay:latest"
COMMAND=(sh -c "cowsay $msg")

APISERVER=https://kubernetes.default.svc
SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount
NAMESPACE=$(cat ${SERVICEACCOUNT}/namespace)
TOKEN=$(cat ${SERVICEACCOUNT}/token)
CACERT=${SERVICEACCOUNT}/ca.crt

kubectl config set-cluster local --server="${APISERVER}" --certificate-authority="${CACERT}"
kubectl config set-credentials local --token="${TOKEN}"
kubectl config set-context local --cluster=local --user=local --namespace="${NAMESPACE}"
kubectl config use-context local

kubectl run task -it --rm --restart=Never --image="$IMAGE" -- "${COMMAND[@]}"

This requires that the windmill-chart serviceaccount (or whatever is running your deployment) needs additional privileges:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: windmill
  name: pod-management
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["pods/attach"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]  
- apiGroups: [""]
  resources: ["events"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pod-management
  namespace: windmill
subjects:
- kind: ServiceAccount
  name: windmill-chart
  namespace: windmill
roleRef:
  kind: Role
  name: pod-management
  apiGroup: rbac.authorization.k8s.io

This might be a bit more generous than absolutely required, but haven't had a chance to fine tune. Apply using kubectl apply -f privileges.yaml

rubenfiszel commented 9 months ago

This is really cool thanks, we will probably make it so that depending on the platform the default docker template change.