tilt-dev / tilt

Define your dev environment as code. For microservice apps on Kubernetes.
https://tilt.dev/
Apache License 2.0
7.57k stars 298 forks source link

Add custom `label` parameter to `k8s_yaml` function for Tiltfile #6106

Closed rrmistry closed 5 months ago

rrmistry commented 1 year ago

Describe the Feature You Want

Add labels parameter to k8s_yaml function.

Current Behavior

Currently, any resource sent to k8s_yaml gets no label. So the resources show up as "unlabeled"

Why Do You Want This?

Similar to https://github.com/tilt-dev/tilt-extensions/issues/299, it would be great to add labels parameter so that objects sent to the function can be customized.

The grouping behavior is very nice. But it would be good to group arbitrary YAMLs together.

This is our user experience:

At this point we pretty much have everything else labeled. And in our minds, just keep a note that anything unlabeled is coming from our Helm chart and its dependencies.

Additional context Our use case is code like this:

# 👍 local_resource(...) has `labels` so can customize
terraform_symbols = load_dynamic(path="terraform/Tiltfile")

# 👍 helm_resource(...) has `labels` so can customize
load('ext://helm_resource', 'helm_resource', 'helm_repo')
helm_chart_dependency_symbols = load_dynamic(path="helm-chart/non-helm-dependencies/Tiltfile")

# 👎 helm(..) does not have `labels` so cannot customize
helm_output = helm(
    '...',
    namespace='...',
    values=['.../values.yaml'],
)

# 👎 k8s_yaml(..) does not have `labels` so cannot customize
k8s_yaml(
  helm_output,
  # labels="Helm Chart" # 👈 Need at least labels for k8s_yaml so it can work with or without `helm`
)
rrmistry commented 1 year ago

Below are suggestions from ChatGPT on how such a requirement could work with tilt.get_resources(...) instead, but as of version 0.32.0, this does not work.

tilt_label = 'my_label'
for resource in tilt.get_resources():
    if resource.name.startswith('your-helm-release-name'):
        k8s_resource(resource.name, labels=[tilt_label])
rrmistry commented 5 months ago

Closing this as a suitable workaround was found.

We are using k8s_resource as a workaround to explicitly define Tilt resources like below:

for w in [..., "my-chart-prometheus", "my-chart-grafana", ...]:
    k8s_resource(workload=w, labels=["infra"]) # <<< Just to define Tilt resource labels

for w in [..., "my-chart-vitess", "my-chart-neo4j", ...]:
    k8s_resource(workload=w, labels=["databases"]) # <<< Just to define Tilt resource labels

for w in [..., "my-chart-api-service1", "my-chart-api-service2", ...]:
    k8s_resource(workload=w, labels=["apis"]) # <<< Just to define Tilt resource labels
... 

Ideally, all the calls to k8s_resource(workload=w, labels=["something"]) would be good to define explicitly within Kubernetes manifests using labels (e.g. tilt.dev/resource: infra) but this workaround above is good enough for now.

rrmistry commented 5 months ago

Found yet another (cleaner) workaround and thought to share it:

# Intercept Tilt resource naming and also apply labels to resources
def apply_resource_labels(id):
    if "prometheus" in id.name:
        # Reference: https://docs.tilt.dev/api.html#api.k8s_resource
        k8s_resource(
            workload=id.name,
            labels=["monitoring"],
        )
    elif "ingress-nginx" in id.name:
        # Reference: https://docs.tilt.dev/api.html#api.k8s_resource
        k8s_resource(
            workload=id.name,
            labels=["ingress"],
        )
    # elif ...
    return id.name

# Reference: https://docs.tilt.dev/api#api.workload_to_resource_function
workload_to_resource_function(apply_resource_labels)
rrmistry commented 5 months ago

Found yet another (cleaner) workaround and thought to share it:

# Reference: https://docs.tilt.dev/api#api.decode_json
tilt_KubernetesApply_list = decode_json(
  # Reference: https://docs.tilt.dev/api#api.local
  json=local(
    command="""
      tilt get KubernetesApplys --output=json
    """,
    echo_off=True,
    quiet=True,
  )
)

for tilt_KubernetesApply in tilt_KubernetesApply_list.get('items', []):
  tilt_KubernetesApply_yaml = tilt_KubernetesApply.get('spec', {}).get('yaml', "")
  for tilt_labels in [l for l in tilt_KubernetesApply_yaml.splitlines() if 'tilt.dev/label' in l]:
    tilt_resource_name = tilt_KubernetesApply.get('metadata', {}).get('name', "")
    tilt_labels = tilt_labels.split(':')[1].strip()
    if tilt_resource_name != None and len(tilt_resource_name) > 0 and tilt_labels != None and len(tilt_labels) > 0:
      # Reference: https://docs.tilt.dev/api#api.k8s_resource
      k8s_resource(
        workload=tilt_resource_name,
        labels=[tilt_labels],
      )

Though, this suffers from the same issue as described in https://github.com/tilt-dev/tilt/issues/6349#issuecomment-2028536755 (where the issue is the above code will update Tilt resource label in the UI on second run, not first)