tilt-dev / tilt

Define your dev environment as code. For microservice apps on Kubernetes.
https://tilt.dev/
Apache License 2.0
7.63k stars 301 forks source link

help iterating on a configmap #3895

Open nicks opened 3 years ago

nicks commented 3 years ago

Currently, if you update a ConfigMap, and some Pod depends on that ConfigMap, Kubernetes will not auto-reload the Pod.

This makes it annoying to iteratively develop on configmaps

There are a few operators in the kubernetes ecosystem that address this use-case, e.g., https://github.com/stakater/Reloader

I could imagine different possible solutions to this problem (e.g., Tilt warning you when a pod is running a stale configmap, or auto-reloading the pod) as well as technical implementations (e.g., there are already operators that do this, maybe Tilt should just auto-install one if it sees that you're using ConfigMaps in your tilt env)

But filing this for now to collect input / thumbs-ups, since this is something I've wanted, and something I've heard users ask about

maiamcc commented 3 years ago

A workaround is to hash the configmap into the YAML of pods/deployments/etc. to be reloaded, a la this example. This is easy with Helm, and awkward but doable with Tilt primitives (read_file, local, string manipulation of YAML or dict manipulation via read_yaml/decode_yaml).

Raboo commented 3 years ago

@maiamcc workaround is quite common. And this is in a sense a kubernetes flaw https://github.com/kubernetes/kubernetes/issues/22368.

Here is my example using yq (v4) and sha256sum:

# Modify .spec.template.metadata.annotations.deploymentHash with a sha256sum of deployment.yaml.
# We need so that the deployment gets restarted when Secrets or ConfigMaps updates
checksum_yaml = local(
    r'''yq -P e ".|select(di == 3) |= .spec.template.metadata.annotations.deploymentHash = \"$(sha256sum deployment.yaml| awk '{print $1}')\"" deployment.yaml''',
    quiet=True)

watch_file('deployment.yaml')
k8s_yaml(checksum_yaml)

However this could be solved easier in the future if there were extensions to Tilt that would a allow a worklfow like this

my_yaml = read_yaml_stream("deployment.yaml")
my_yaml[3]["spec"]["template"]["metadata"]["annotations"]["deploymentHash"] = str(local("sha256sum deployment.yaml"))
k8s_yaml(encode_yaml_stream(my_yaml))

As per this slack conversation https://kubernetes.slack.com/archives/CESBL84MV/p1616187529029900?thread_ts=1616151635.024300&cid=CESBL84MV

landism commented 3 years ago

fwiw, this applies to secrets as well