live_update doesn't work on `scratch`

[landism]

Steps to Reproduce

1. Write a Tiltfile with a docker_build that starts with FROM scratch
2. Configure it with live_update You can test in the 3-recommended directory of this branch

Expected Behavior

live_update works, or Tilt gives an error message saying it's not supported

Current Behavior

Live Update failed with unexpected error:
    copying changed files: Internal error occurred: error executing command in container: failed to exec in container: failed to start exec "451e7bb1a3654e5e0a10fb8a0334987f06884dd885ddd059f9afa615293a48be": OCI runtime exec failed: exec failed: container_linux.go:349: starting container process caused "exec: \"tar\": executable file not found in $PATH": unknown
Falling back to a full image build + deploy

Notes

The two obvious options are:

1. Have Tilt stick its own copy of tar into the container.
2. Find some non-tar way of syncing files into a container.

kubectl cp sounds promising, but...:

$ kubectl cp --help
Copy files and directories to and from containers.

Examples:
  # !!!Important Note!!!
  # Requires that the 'tar' binary is present in your container
  # image.  If 'tar' is not present, 'kubectl cp' will fail.

[nicks]

Lots of prior discussion on this here: https://github.com/kubernetes/kubernetes/issues/58512. It comes up a lot when you're using Bazel or Ko distroless images. I usually show people how to use a different base image that's better for live development

i would probably call this issue a feature request rather than a bug, given that Kubernetes treats it as kind/feature.

[mikesouza]

Lots of prior discussion on this here: kubernetes/kubernetes#58512. It comes up a lot when you're using Bazel or Ko distroless images. I usually show people how to use a different base image that's better for live development

i would probably call this issue a feature request rather than a bug, given that Kubernetes treats it as kind/feature.

@nicks What would you recommend as a better base image to use for live development?

[milas]

The main requirements are sh and tar. You could use the busybox image, for example: https://hub.docker.com/_/busybox which is probably the smallest image with minimal replacements of most Unix tools.

To use with Tilt, your Dockerfile would be something like:

ARG base_image=scratch
FROM $base_image

...

Then in your Tiltfile:

docker_build_with_restart(
  'my-image',
  '.',
  dockerfile='Dockerfile',
  build_args={'base_image': 'busybox'},
  live_update=[
    ...
  ]
)

The GoogleContainerTools/distroless images are also popular minimal images, which, as @nicks mentioned, lack a shell/tools by default, but publish debug tags, e.g. gcr.io/distroless/base:debug that include busybox and would be suitable.