DNS customizations not applied within kubernetes when vm-type=vz.

Description

Containers run under Kubernetes cannot resolve host.docker.internal nor custom addresses specified using the --dns-host flag. This only happens within Kubernetes; similar containers running at the Docker layer have no problems resolving these addresses. This only happens when --vm-type=vz is specified; DNS resolution works as expected when --vm-type=qemu is used.

Version

colima version v0.6.7 git commit: ba1be00e9aec47f2c1ffdacfb7e428e465f0b58a WARN[0000] cannot detect current running executable: error looking up 'colima-Darwin-arm64' in PATH: exec: "colima-Darwin-arm64": executable file not found in $PATH WARN[0000] falling back to first CLI argument

runtime: docker arch: aarch64 client: v23.0.5 server: v24.0.7

kubernetes Client Version: v1.26.13 Kustomize Version: v4.5.7 Server Version: v1.28.3+k3s2 limactl version 0.19.1 qemu-img version 8.2.0 Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers

Operating System

[ ] macOS Intel <= 13 (Ventura)
[ ] macOS Intel >= 14 (Sonoma)
[ ] Apple Silicon <= 13 (Ventura)
[X] Apple Silicon >= 14 (Sonoma)
[ ] Linux

Output of `colima status`

WARN[0000] cannot detect current running executable: error looking up 'colima-Darwin-arm64' in PATH: exec: "colima-Darwin-arm64": executable file not found in $PATH WARN[0000] falling back to first CLI argument
INFO[0000] colima is running using macOS Virtualization.Framework INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] address: 192.168.107.2
INFO[0000] socket: unix:///Users/peter/.colima/default/docker.sock INFO[0000] kubernetes: enabled

Reproduction Steps

Start from a fresh state: run colima delete to clear any lingering state. Maybe not necessary, but better to be sure.

Start Colima specifying a VZ VM. Optionally include additional DNS hosts.

colima start --arch aarch64 --vm-type=vz --vz-rosetta --mount-type=virtiofs --memory=16 --with-kubernetes --network-address --dns-host 'foo.bar=192.168.5.2'

Pull an Alpine image for testing
```
docker pull alpine:3.19
```

Create a dummy pod we can kubectl-exec into based on the Alpine image

kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
name: alpine
namespace: default
spec:
containers:
- image: alpine:3.19
command:
  - /bin/sh
  - "-c"
  - "sleep 60m"
imagePullPolicy: Never
name: alpine
restartPolicy: Always
EOF

Kubectl-exec into the pod and try resolving addresses.


❮ kube exec -it alpine -- nslookup host.docker.internal
Server:     10.43.0.10
Address:    10.43.0.10:53

** server can't find host.docker.internal: NXDOMAIN

command terminated with exit code 1 ❯ kube exec -it alpine -- nslookup foo.bar Server: 10.43.0.10 Address: 10.43.0.10:53

Non-authoritative answer:

❯ kube exec -it alpine -- nslookup github.com Server: 10.43.0.10 Address: 10.43.0.10:53

Non-authoritative answer:

Non-authoritative answer: Name: github.com Address: 192.30.255.113


`host.docker.internal` fails with `NXDOMAIN`; our injected `foo.bar` comes back with no results; resolution of ordinary domains such as `github.com` work as normal.

6. Restart VM; repeat DNS requests using Alpine Docker container. Restart seems to be needed otherwise Docker resolutions sometimes fail too for a short while (cacheing maybe?).

❮ docker run -it alpine:3.19 nslookup host.docker.internal
Server: 192.168.5.2 Address: 192.168.5.2:53

Non-authoritative answer:

Non-authoritative answer: Name: host.docker.internal Address: 192.168.5.2

❯ docker run -it alpine:3.19 nslookup foo.bar Server: 192.168.5.2 Address: 192.168.5.2:53

Non-authoritative answer:

Non-authoritative answer: Name: foo.bar Address: 192.168.5.2

❯ docker run -it alpine:3.19 nslookup github.com Server: 192.168.5.2 Address: 192.168.5.2:53

Non-authoritative answer:

Non-authoritative answer: Name: github.com Address: 192.30.255.113


Resolutions seem to be working at the Docker layer. Our pod is still around, we can try the resolutions there again, but they still fail.

7. Run the above steps again, but under QEMU. All resolutions succeed.

colima start --arch aarch64 --vm-type=qemu --memory=16 --with-kubernetes --network-address --dns-host 'foo.bar=192.168.5.2'



### Expected behaviour

My expectations had been:
* resolution works the same for all containers, regardless of VM type, regardless of whether run under Kubernetes or not
* `host.docker.internal` resolves to `192.168.5.2`
* custom DNS results specified using `--dns-host` apply to all containers. In the above case `foo.bar` returns a result of `192.168.5.2`.

### Additional context

_No response_

abiosoft / colima