Add ability to ignore "Image not used in any Kubernetes config" warning

ankeesler commented 3 years ago

Describe the Feature You Want

I would like to tell tilt that I want to build a Docker image with hot restart capabilities (i.e., docker_build_with_restart()), even though I am not using that image in a spec.containers[].image field.
Instead I am going to use that image name in another piece of Kube YAML (e.g., ConfigMap) to pass to an operator to deploy a Pod with that image that I built.

Current Behavior

When I tell tilt to build a Docker image (i.e., docker_build_with_restart()) that I don't use in any spec.containers[].image field, it doesn't build the image and gives me this helpful warning.
```
Image not used in any Kubernetes config:
✕ image/kubecertagent
Did you mean…
...
Skipping this image build
```
Then, when my operator tries to deploy a Pod with that image, it can't since the image doesn't exist, and the Pod goes into ImagePullBackOff.

Why Do You Want This?

We (https://github.com/vmware-tanzu/pinniped) are using tilt to iterate quickly on local development (i.e., make a change in *.go code, and see that change show up in a local kind cluster within 5 seconds or so).

Right now, we are working around this issue by using the same image for our deployed Pod as we are for the operator, but we see this error and get reload times of more like 15 seconds.

Will not perform Live Update because:
Error retrieving container info: can only get container info for a single pod; image target image:image/concierge has 2 pods
Falling back to a full image build + deploy

TL;DR - we would like to speed up that 15 second reload time to the normal ~5 seconds.

Additional context

Here are some snippets of our Tiltfile. Notice 1) the attempt to build the image/kubecertagent image with a docker_build_with_restart() and 2) the attempt to pass it as a ytt parameter below. Our Concierge app will consume that kube_cert_agent_image template parameter via a ConfigMap. It reads the ConfigMap and then deploys a Pod with the image name provided.

...
docker_build_with_restart('image/kubecertagent', '.',
    dockerfile='kubecertagent.Dockerfile',
    entrypoint=['/usr/local/bin/pinniped-concierge'],
    live_update=[sync('./build/pinniped-concierge', '/usr/local/bin/pinniped-concierge')],
    only=['./build/pinniped-concierge'],
)
...
k8s_yaml(local([
    'sh', '-c',
    'ytt --file ../../../deploy/concierge ' +
    '--data-value app_name=pinniped-concierge ' +
    ...
    '--data-value-yaml kube_cert_agent_image=image/kubecertagent:tilt-dev',
]))

ankeesler commented 3 years ago

This seems related to https://github.com/tilt-dev/tilt/issues/3732.

nicks commented 3 years ago

Unfortunately, even if you turned off the warning, the deploy still wouldn't work. :\

There's two separate problems:

Tilt gives each image a content-based tag, and injects that tag into the YAML
Tilt has to know which YAML to deploy when the image changes so for both of these, it needs to be able to find the image in the YAML.

Fortunately, we have a workaround for cases like this! See this guide: https://docs.tilt.dev/custom_resource.html which tells you how to use Tilt's APIs to tell it where to inject the image, and has some examples (e.g., Airflow env variables)

ankeesler commented 3 years ago

Wow, thanks @nicks ! That's really helpful. I'll give that a try. Thanks for the quick turnaround!

bshore commented 2 years ago

@nicks Hope it's okay to do this, but I'll share a workaround I ended up using after multiple searches brought me here:

My Tiltfile builds the Dockerfile

docker_build(
    'my-image-name',
    '.',
    entrypoint=["echo", "this", "is", "a", "no", "op", "pod"],
    dockerfile='/Dockerfile',
    only=['/some/code'],
    live_update=[
        sync('/some/code', '/code'),
    ],
    extra_tag=["my-image-name:local"], # <- Tilt still does it's internal tags, but also
                                       # creates this tag that can be used outside of Tilt
)

I use a no-op Job/Pod that gets pulled in as part of k8s_yaml(kustomize()) in a file named no-op.yaml.

# This is a no op Job, we're doing this so we can gain the benefits of Tilt
# auto-building the local image for us
---
apiVersion: batch/v1
kind: Job
metadata:
  name: no-op
  namespace: default
spec:
  template:
    spec:
      containers:
      - name: no-op
        image: my-image-name
      restartPolicy: Never

I can then do things like docker run my-image-name:local echo "Hello" or manually apply kubectl apply -f non-tilt.yaml that would use this image

tilt-dev / tilt